-- Logs begin at Wed 2022-08-31 09:00:16 UTC, end at Wed 2022-08-31 10:13:22 UTC. -- Aug 31 09:32:03.119505 dv-cinder-837542-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 09:32:03.126816 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 09:32:03 2022] *** Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: nodename: dv-cinder-837542-8-iscsi Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: machine: x86_64 Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: clock source: unix Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: pcre jit disabled Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: detected number of CPU cores: 4 Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: current working directory: / Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: detected binary path: /usr/bin/uwsgi-core Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: your processes number limit is 47680 Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: your memory page size is 4096 bytes Aug 31 09:32:03.128846 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: detected max file descriptor number: 2048 Aug 31 09:32:03.129573 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: lock engine: pthread robust mutexes Aug 31 09:32:03.131428 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: thunder lock: enabled Aug 31 09:32:03.131428 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 09:32:03.131428 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 09:32:03.169717 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: Python main interpreter initialized at 0x55be0b83a9c0 Aug 31 09:32:03.169717 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: python threads support enabled Aug 31 09:32:03.169717 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: your server socket listen backlog is limited to 100 connections Aug 31 09:32:03.169717 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: your mercy for graceful operations on workers is 90 seconds Aug 31 09:32:03.170089 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 09:32:03.170158 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: *** Operational MODE: preforking *** Aug 31 09:32:03.170398 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: *** uWSGI is running in multiple interpreter mode *** Aug 31 09:32:03.170398 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: spawned uWSGI master process (pid: 88227) Aug 31 09:32:03.170599 dv-cinder-837542-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 09:32:03.173340 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: spawned uWSGI worker 1 (pid: 88241, cores: 1) Aug 31 09:32:03.173340 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: spawned uWSGI worker 2 (pid: 88242, cores: 1) Aug 31 09:32:03.173340 dv-cinder-837542-8-iscsi devstack@keystone.service[88227]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 09:32:07.491677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:07.491677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: from cryptography.utils import int_from_bytes Aug 31 09:32:07.492450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:07.492450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: from cryptography.utils import int_from_bytes Aug 31 09:32:07.492875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:07.492875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: from cryptography.utils import int_from_bytes Aug 31 09:32:07.493112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:07.493112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: from cryptography.utils import int_from_bytes Aug 31 09:32:07.541484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:32:07.541674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:32:07.541865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] command line args: [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:32:07.542135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:32:07.542311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ================================================================================ {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:32:07.542398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:32:07.542564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] admin_token = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.542656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:32:07.542772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] command line args: [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:32:07.542851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] config_dir = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.542982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:32:07.543148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] config_source = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ================================================================================ {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:32:07.543327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] debug = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] admin_token = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] config_dir = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.543969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] config_source = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] debug = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.544875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_config_append = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_dir = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_file = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_config_append = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.545836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_dir = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_file = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.546945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.547984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.548926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] publish_errors = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.549905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] publish_errors = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] standard_threads = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.550895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] use_journal = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] standard_threads = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] use_json = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] use_stderr = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.551997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] use_syslog = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] use_journal = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.552656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] use_json = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.552809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.552890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] use_stderr = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.553049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] use_syslog = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.553126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.553239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:07.553335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.553501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.553587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.553744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.553824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.554975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.555939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.556905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.557928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.558964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.559920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.560876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.561983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.562062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.562215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.562784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.562958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.563963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.564890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.565888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.566952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.567962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.568815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.569948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.570891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.571967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.572919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.573970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.574937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.575873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.576939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577195 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.577947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.578542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.578711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.578790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.578954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.external = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.579938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.password = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.external = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] auth.token = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.580880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.password = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] auth.token = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.581941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.582902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.583868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.584827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.585833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.586979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.587923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.588983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.589972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.590912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.591868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.592999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.593926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.594961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.595991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.596939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.597913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.598850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.599949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.600898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.601946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.602895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.603990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.604926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.605978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.606924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.607864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.608993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.609872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.610845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.611970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.612932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] role.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] role.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613508 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.613998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] role.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] role.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.614951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.615918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.616801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.617951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.618987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.619923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.620889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.621888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.622986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.623961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.624680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.625949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.connection = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.626934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.connection = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.627966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.628916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.629898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.630950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:32:07.631176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.631942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:07.632114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88241) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:32:07.660276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.660503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.660699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.660794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.660930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.661175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.668999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.669614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.697347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.697575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.697999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.698233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.750547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.750913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.798618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.798839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.799487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.799726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.954434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.954659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.954886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.955108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.955323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.955536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.955628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.955855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.956082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.956304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.956525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.956744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.978983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.979113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.979198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.979323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.979406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.979533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.991069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:07.996768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.006138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.006478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.006881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.007277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.007666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.008044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.008416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.010623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.010864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.011088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.011311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.011530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.011751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.011968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88241) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:08.025813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.026026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:08.026287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.026371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.026500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:08.026583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:08.026813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.026896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.027031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:08.027113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.027311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.027402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.027562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.027819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.028088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.028351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.037232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.037515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.037787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.037869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.038071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.038155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.038373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.038657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.045041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.045664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.048450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.048736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.049336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.049962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.050963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.051054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.051224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.051344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.051624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.051899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.074554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.075288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.076010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.076296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.076762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.077080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.080856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.081166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.081450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.081787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.082077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.082369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.087908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.088198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.088711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.089029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.092296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.092636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.092929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.093150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.093255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.093448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.093538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.093771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.094046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.094323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.094600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.094873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.095145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.095572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.095862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.096143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.112281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.112562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.112733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.113043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.116438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.116891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.119541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.119839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.120141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.120441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.124167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.124644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.127894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.128425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.131025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.131341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.131648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.131730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.131931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.132815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.133125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.148345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.148635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.148903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.149017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.149194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.149277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.149471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.149739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.155926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.156645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.157638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.158201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.159180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.159458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.159742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.160024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.162134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.162423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.162702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.162786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.162982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.163066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.163284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.163369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.163639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.163936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.175111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.175397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.175620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.175719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.175910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.175995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.176877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.177120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.177408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.192563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.193367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.195849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.196548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.198752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.199464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.200183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.200913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.201679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.201963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.202244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.202543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.202834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.203148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.208719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.209578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.210212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.211064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.213091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.213399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.213683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.213956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.214116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.214412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.214710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.214989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.221964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.222254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.222531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.222942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.223236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.223521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.229408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.229693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.229971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.230245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.230429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88241) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:08.230527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.230718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.230802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.231957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.232905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.233010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.233202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.233465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.233732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.234011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88241) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:08.267161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be0b83a9c0 pid: 88242 (default app) Aug 31 09:32:08.268533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be0b83a9c0 pid: 88241 (default app) Aug 31 09:32:08.270683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbb72-5b6c-45a3-bb75-02d37b857090 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:08.271025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbb72-5b6c-45a3-bb75-02d37b857090 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:08.271354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbb72-5b6c-45a3-bb75-02d37b857090 None None] PATH_INFO: `/v3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:08.275540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Wed Aug 31 09:32:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:14.830466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:14.830982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:14.831086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:14.835448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:14.840622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:14.841109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:14.841239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:14.910997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: INFO dbcounter [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] Registered counter for database keystone Aug 31 09:32:14.929518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG oslo_db.sqlalchemy.engines [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:32:14.935694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writer thread running {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:32:15.114394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG passlib.handlers.bcrypt [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88242) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:32:15.114785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG passlib.handlers.bcrypt [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88242) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:32:15.173037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:15.174618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:15.307565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 09:32:15.756799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:15.826669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: INFO dbcounter [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] Registered counter for database keystone Aug 31 09:32:15.848022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG oslo_db.sqlalchemy.engines [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88241) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:32:15.854191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writer thread running {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:32:15.975227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1ca7e-a776-4700-8268-2ae8a2f5890c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:15.979478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:15.980100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:15.980387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:15.980666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:15.990245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:16.046771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:32:16.047108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde0d21-a092-4168-add6-41968abc5b17 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:16.057660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Aug 31 09:32:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:32:16.063624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:16.078306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d0ab-11fd-4e35-91ce-6b9d0ed576a0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:16.082923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:16.084074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:16.084360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:16.084641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] PATH_INFO: `/v3/projects/908a4dee0a684e749e89f40e4287b799` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:16.086249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] RBAC: Authorizing `identity:get_project(project_id=908a4dee0a684e749e89f40e4287b799)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:16.139822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:32:16.140180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d04e51-7a27-4c06-a26d-9e6ea1c62fb9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:16.145332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Wed Aug 31 09:32:16 2022] GET /identity/v3/projects/908a4dee0a684e749e89f40e4287b799 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:18.036316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11458a45-5a83-4271-8947-05fe1b6cf71e None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:18.036926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11458a45-5a83-4271-8947-05fe1b6cf71e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:18.036926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11458a45-5a83-4271-8947-05fe1b6cf71e None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:18.041044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:18.046593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:18.047019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:18.047195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:18.067702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:18.069271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:18.077410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 09:32:18.352886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11458a45-5a83-4271-8947-05fe1b6cf71e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:18.365538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11458a45-5a83-4271-8947-05fe1b6cf71e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:18.369784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:18.370385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:18.370668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:18.370942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:18.378579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:18.380356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f943e-59b9-4f6c-a254-5cf0f8df24f6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:18.389025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Wed Aug 31 09:32:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:32:18.394847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:18.403959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f6fce-e5f7-4af0-8169-d38616273214 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:18.408834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:18.409495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:18.409782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:18.410061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:18.411370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:18.413242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9329d2-39c4-47b3-9571-716f431edf14 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:18.414762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Wed Aug 31 09:32:18 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:20.355300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269597de-a212-4ea3-b1b1-78d159b801fa None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:20.355796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269597de-a212-4ea3-b1b1-78d159b801fa None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:20.355796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269597de-a212-4ea3-b1b1-78d159b801fa None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:20.357118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:20.364521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:20.364985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:20.365144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:20.385057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:20.386275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:20.419473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531624e-3818-4b8d-98b6-c4887e0104ec None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:20.419864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531624e-3818-4b8d-98b6-c4887e0104ec None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:20.420213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531624e-3818-4b8d-98b6-c4887e0104ec None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:20.421606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:20.427583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:20.427778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:20.428124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:20.467455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:20.470196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG passlib.handlers.bcrypt [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88241) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:32:20.470600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG passlib.handlers.bcrypt [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88241) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:32:20.523213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:20.524819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:20.624851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:20.751986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:20.777247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:20.864509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d6bdb-c7fa-43ef-bd59-cfdf3906f261 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:20.897451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:20.899218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:20.899866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:20.900144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:20.900423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:20.912411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:20.921322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:20.922279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:20.923116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:20.928427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.013027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b432ce-4485-4c03-99b2-240d077dbc5e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.015397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.021840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.044630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ad7a-4940-433a-89ea-1a4567ed4775 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.058202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.059118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.059645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.059991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.091092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.092135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.092429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.092849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.093586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:21.093924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.095330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.109439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:21.115356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.139686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.144038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.229503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94caea-6493-485e-9518-b18c52edba4e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.252683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:32:21.258753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.261599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.270628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.271331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.271622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.271903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.273284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.274761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.276654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:21 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:21.282265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.301656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.365767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e199-2afb-4148-9858-72b3fe8d4a9f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.395721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917d03-55d7-4265-9495-2a00235761c6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.406207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.407348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.407627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.407902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.427448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.428947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.435018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.435945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.436406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.436874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.438889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.440777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.442307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:21.448099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:32:21.449385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:21.450224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:32:21.451029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:32:21.451029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:21.451029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:21.451029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:21.456255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:21.518226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.542826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:21.566997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8444e-62ba-4e3f-a8d0-3471de15ee80 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.570674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1592ca-1a00-4148-905c-b276b591e89b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:21.604758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.605444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.605727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.606005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.606738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.domain.id=default, user.id=3c3232bfea3b429697a9f4a4bfb3705a) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:21.607041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.608293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:21.608516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871aafa3-d5f1-42c0-a40b-843563ce54a1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.608912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:21.609233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:21.610110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:21.610765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:21.611077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:21.612629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41dae5-11aa-49eb-a2b4-7eea66dda344 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:21.616585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Wed Aug 31 09:32:21 2022] GET /identity/v3/role_assignments?user.id=3c3232bfea3b429697a9f4a4bfb3705a&scope.domain.id=default&role.id=3058b83cad264d36bc1ba356d7561bef => generated 233 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:21.633325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:23.984086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f580-e7f0-4600-81c6-ae61aa983447 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:23.985175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f580-e7f0-4600-81c6-ae61aa983447 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:23.985252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f580-e7f0-4600-81c6-ae61aa983447 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:23.986828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:23.992126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9ff8c-048f-4f23-8148-c2e39870c791 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:23.992561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9ff8c-048f-4f23-8148-c2e39870c791 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:23.992748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9ff8c-048f-4f23-8148-c2e39870c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:24.025985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-44a9ff8c-048f-4f23-8148-c2e39870c791 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:24.027217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-44a9ff8c-048f-4f23-8148-c2e39870c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:24.080985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22829-4869-4260-8430-d1c0d34b7c05 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:24.084991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22829-4869-4260-8430-d1c0d34b7c05 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:24.084991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22829-4869-4260-8430-d1c0d34b7c05 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:24.085698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:24.094753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946416d9-4974-48ce-9a71-0d7f1b97e7af None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:24.095206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946416d9-4974-48ce-9a71-0d7f1b97e7af None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:24.096781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946416d9-4974-48ce-9a71-0d7f1b97e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:24.136196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-946416d9-4974-48ce-9a71-0d7f1b97e7af None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:24.137772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-946416d9-4974-48ce-9a71-0d7f1b97e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:24.222268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:24.327323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:24.614777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c909-6faf-4fb7-a34f-0e2359465de4 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:24.616620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c909-6faf-4fb7-a34f-0e2359465de4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:24.616620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2c909-6faf-4fb7-a34f-0e2359465de4 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:24.616620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 12/26] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:24.622327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:24.622628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:24.622913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:24.661957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:24.663180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:24.686389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2c909-6faf-4fb7-a34f-0e2359465de4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:24.786354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:24.892128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 15/27] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:24.893968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:24.898264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:24.997682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2c909-6faf-4fb7-a34f-0e2359465de4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.102232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.103102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.103576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.103993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.139686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:25.148744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:25.149644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:25.150418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:25.155187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 13/28] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:25.162632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:25.183794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5cafc-6d5b-4380-a966-75f09b405970 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.251788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.252016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.253103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.253103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.270398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:25.273137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c6aa2872-8ee8-4182-9261-5f77ce93e658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:32:25.291825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:25.292714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:32:25.293588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:32:25.293588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:25.293588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:25.295377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 16/29] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:25.296889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6425-0dc0-4d9e-a36f-397d7579a234 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.297843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6425-0dc0-4d9e-a36f-397d7579a234 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.298330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6425-0dc0-4d9e-a36f-397d7579a234 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.299849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 17/30] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:25.306116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d6425-0dc0-4d9e-a36f-397d7579a234 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:25.396770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:25.435439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345da882-eb99-4db3-a887-239ddceccf61 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.444033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:25.456530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d6425-0dc0-4d9e-a36f-397d7579a234 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.507942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.508583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.508872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.509169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.509826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:25.510145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:25.513937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:25.519656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.520301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.520603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.520859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.521576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:25.521885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:25.523299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32474-1439-4efc-9700-747da9168e36 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:25.551644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 14/31] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:25.553875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.554169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.554444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.560275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 18/32] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:25.563675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:25.592222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:25.593464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:25.705061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 15/33] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:25.705061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:25.722414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717766f-019e-4fe9-8bfb-f6c62c04d7da None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.744333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:25.775564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.776373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.776660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.776939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.777654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:25.779060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afacb7-d1aa-4bfa-82c0-27484ca255cb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:25.877504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:25.882783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2dcc2-43c2-4784-b987-400ce01c72e2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:25.935540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:25.936436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.936860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.938090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.939842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:25.941698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:25.943924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 16/34] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:25 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:25.945307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:25.987398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 19/35] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:25 2022] POST /identity/v3/roles => generated 219 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:25.988840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8fa27-b73f-4ff6-97fa-d29a3d646376 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:25.991263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8fa27-b73f-4ff6-97fa-d29a3d646376 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:25.991574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8fa27-b73f-4ff6-97fa-d29a3d646376 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:25.993213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 20/36] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:25.993790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8fa27-b73f-4ff6-97fa-d29a3d646376 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:26.086305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03a4e0-0801-4263-8023-91521b75ff90 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:26.131429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:26.132322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.132702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.133151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.133370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8fa27-b73f-4ff6-97fa-d29a3d646376 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:26.133812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:26.135278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1152fdd-b688-4a47-94c6-2fe5a6e8f42e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:26.181462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:26.182184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.182480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.182758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.184475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:26.186272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:26.194127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 21/37] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:26.194127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:26.246698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:26.294235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 17/38] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:32:25 2022] POST /identity/v3/projects => generated 301 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:26.294235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532beb65-dbbf-4ee3-bcf8-e1994c4133a5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.294235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532beb65-dbbf-4ee3-bcf8-e1994c4133a5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.294235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532beb65-dbbf-4ee3-bcf8-e1994c4133a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.318219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f23a6-dac0-40b3-ae06-1e8ad3bd1aaf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:26.322198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-532beb65-dbbf-4ee3-bcf8-e1994c4133a5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:26.329258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-532beb65-dbbf-4ee3-bcf8-e1994c4133a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:26.346341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:26.366360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:26.367000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.367286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.367563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.368259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:26.369717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b098b2-83a8-42c7-94de-eb819a39e263 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:26.397511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 22/39] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:26 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:26.398908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789272e0-afad-4977-af7f-27ff76070a13 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.399203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789272e0-afad-4977-af7f-27ff76070a13 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.399596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789272e0-afad-4977-af7f-27ff76070a13 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.400829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 23/40] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:26.401578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789272e0-afad-4977-af7f-27ff76070a13 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:26.421810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 18/41] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:26.423269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.427020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.427020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.469284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:26.475355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:26.615551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 19/42] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:26.634350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789272e0-afad-4977-af7f-27ff76070a13 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:26.636924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:26.737000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:26.793982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:26.794664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.794954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.796241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] PATH_INFO: `/v3/domains/default/users/3c3232bfea3b429697a9f4a4bfb3705a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.797321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3c3232bfea3b429697a9f4a4bfb3705a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:26.798788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:26.817003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 24/43] 10.222.0.25 () {58 vars in 1361 bytes} [Wed Aug 31 09:32:26 2022] PUT /identity/v3/domains/default/users/3c3232bfea3b429697a9f4a4bfb3705a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:26.854283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:26.879504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0d83-716b-4a46-9574-4dcb652f01c4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:26.919286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:26.919927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:26.920213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:26.920493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:26.921332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:26.922107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:26.925126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:26.948061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 20/44] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:26 2022] POST /identity/v3/roles => generated 223 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:27.060839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05c06c-ef09-4ad5-9d41-cb3b5969be35 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:27.112448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:27.114075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:27.114075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:27.114075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:27.116061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:27.117797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31aa8a7-92cb-4e96-8eaf-0a9bb6f0fcf1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:27.120189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 25/45] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:27.126351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:27.247502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4378e-f5f6-4763-8748-4245152c4bd6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:27.283330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:27.283970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:27.284258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:27.284538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:27.285258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:27.286708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f7b37-65f0-464a-8826-95ef59972f8a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:27.297651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:27.315990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 21/46] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:32:27 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:28.785019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14703b8-d3b4-4c84-a133-6234fa762c69 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:28.785515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14703b8-d3b4-4c84-a133-6234fa762c69 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:28.785611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14703b8-d3b4-4c84-a133-6234fa762c69 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:28.787073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 26/47] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:28.792300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f70a-d93d-42af-b1f0-9044a2d1bdf2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:28.792711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f70a-d93d-42af-b1f0-9044a2d1bdf2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:28.792867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f70a-d93d-42af-b1f0-9044a2d1bdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:28.811826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0302f70a-d93d-42af-b1f0-9044a2d1bdf2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:28.813202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0302f70a-d93d-42af-b1f0-9044a2d1bdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:28.884829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 22/48] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:29.153074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14703b8-d3b4-4c84-a133-6234fa762c69 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.186255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:29.251618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da74d2-cb01-416c-a7fe-2235db51e6af None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.252408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da74d2-cb01-416c-a7fe-2235db51e6af None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.252408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da74d2-cb01-416c-a7fe-2235db51e6af None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.257013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 23/49] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:29.262682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.262966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.263241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.275867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14703b8-d3b4-4c84-a133-6234fa762c69 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.284263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:29.285500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:29.316326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.317014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.317293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.317574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.329604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:29.335454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:29.336359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:29.337171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 27/50] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:29.343443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.369139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 24/51] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:29.441361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:29.451232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14f5b2-2437-44ad-8efd-02b524f0f939 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.488810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.489494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.489782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.490060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.490732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:29.491040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:29.492475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:29.501119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 28/52] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:29.506850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.563049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:29.617248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9659-5090-4a73-bb72-e17c13437211 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.632853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.658843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.659487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.659770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.660049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.661226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:29.662672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:29.664408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 25/53] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:29.669884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.748650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d6bb-84a2-4782-a28d-052bc0da5662 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.793439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.794455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.794897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.795353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.797513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:29.800185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:29.802383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 29/54] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:29.808245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.838719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:29.901107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff523d9d-cab7-4ebc-9c23-a17c01ce9a56 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.919192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:29.921331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4506-e7ea-4bc7-ba50-b457467164c2 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:29.937906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.938535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.938819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.939103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.940190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:29.941641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:29.943378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 26/55] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:29.950709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:29.963385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:29.964313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:29.964610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:29.964893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:29.965590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:29.967219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9ace1-9f39-4256-96a8-44e67dfe0288 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:30.002412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 30/56] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Aug 31 09:32:29 2022] POST /identity/v3/projects => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:30.069621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafffb7-d71a-4672-9346-0af8479f1b47 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:30.114311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:30.114942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:30.115236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:30.115521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:30.116271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.domain.id=default, user.id=3c3232bfea3b429697a9f4a4bfb3705a) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:30.116587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:30.118158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f95e6-5341-497a-8086-c277b40576c1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:30.130201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 27/57] 10.222.0.25 () {56 vars in 1322 bytes} [Wed Aug 31 09:32:29 2022] GET /identity/v3/role_assignments?user.id=3c3232bfea3b429697a9f4a4bfb3705a&scope.domain.id=default&role.id=3058b83cad264d36bc1ba356d7561bef => generated 530 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:31.931221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdadec7-9a36-4a16-9201-1536f7d22862 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:31.931766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdadec7-9a36-4a16-9201-1536f7d22862 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:31.931766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdadec7-9a36-4a16-9201-1536f7d22862 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:31.933158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 31/58] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:31.938352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:31.938724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:31.938914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:31.957374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:31.958609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:32.003404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:32.023988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 28/59] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:32.291786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdadec7-9a36-4a16-9201-1536f7d22862 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:32.369274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:32.397404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdadec7-9a36-4a16-9201-1536f7d22862 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:32.431018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:32.431659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:32.431953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:32.432229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:32.433383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:32.434837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35376624-db6a-4b75-9f7e-3e7b2a44e64a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:32.436399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 32/60] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:32.442480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:32.542420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc951ff-5795-420a-9546-8abc09dbad46 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:32.576007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:32.576623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:32.576908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:32.577218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:32.577861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:32.579365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3d4da-41d2-4b79-97ef-69d94b82642b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:32.605254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 29/61] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:32 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:35.673800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1c69-c04e-426b-bed1-0dbba6e19f42 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.674690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1c69-c04e-426b-bed1-0dbba6e19f42 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.675125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1c69-c04e-426b-bed1-0dbba6e19f42 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.676667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 33/62] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:35.682137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.682977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.683402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.695293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383119e8-fe52-48cf-b641-166f69b8ee8b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.695293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383119e8-fe52-48cf-b641-166f69b8ee8b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.695293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383119e8-fe52-48cf-b641-166f69b8ee8b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.695293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 34/63] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:35.700598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40964482-3e5e-44fc-9e86-851bb8060e98 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.701193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40964482-3e5e-44fc-9e86-851bb8060e98 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.701673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40964482-3e5e-44fc-9e86-851bb8060e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.715038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:35.717010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:35.721681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-40964482-3e5e-44fc-9e86-851bb8060e98 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:35.723396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-40964482-3e5e-44fc-9e86-851bb8060e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:35.785350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:35.821161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 35/64] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:35.822438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fca61-b0ea-4ef6-baef-e3b83baeab0e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.825294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fca61-b0ea-4ef6-baef-e3b83baeab0e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.825591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fca61-b0ea-4ef6-baef-e3b83baeab0e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.827163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 36/65] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:35.833101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 30/66] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:35.834323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2843f6b-ca19-48fa-9245-c24c1ba79663 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:35.834616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2843f6b-ca19-48fa-9245-c24c1ba79663 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:35.834901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2843f6b-ca19-48fa-9245-c24c1ba79663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:35.853768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c2843f6b-ca19-48fa-9245-c24c1ba79663 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:35.854968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c2843f6b-ca19-48fa-9245-c24c1ba79663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:35.923884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 37/67] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:36.177395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.237395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af559c-efd3-416b-8ba1-3f599b6c956e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.237906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af559c-efd3-416b-8ba1-3f599b6c956e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.238357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af559c-efd3-416b-8ba1-3f599b6c956e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.239841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 38/68] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:36.244745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.245280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.245760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.252883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.271077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:36.272818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:36.308396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d910ab5-4941-4c4e-82e4-ec68c394b1d1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.346056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.346427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 39/69] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:36.347274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.353295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.353843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.354331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.368789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:36.376592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:36.377515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:36.378285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 31/70] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:36.378917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.416625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.456411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62a713-62a4-4d6e-b29f-67e8163b8fff None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.488457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.493919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.494948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.495214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.495462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.503286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d487-f993-461f-98eb-c5cb1f430b34 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.506975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-3f271688-e0e8-40c4-a90b-ac67e05436b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:36.511905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:36.512757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:36.513603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 40/71] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:36.515504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10705d-04aa-46ed-b747-7fd46465364c None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.516077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10705d-04aa-46ed-b747-7fd46465364c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.516429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10705d-04aa-46ed-b747-7fd46465364c None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.517923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 41/72] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:36.518583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10705d-04aa-46ed-b747-7fd46465364c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.543475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.544367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.544850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.545453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.557356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:36.562369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:36.563864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:36.565316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 32/73] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:36.569436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.629392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10705d-04aa-46ed-b747-7fd46465364c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.658104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.663469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.664120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.664406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.664739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.665434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:36.665742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:36.667168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099efed-f813-46fa-80ca-4c47355bf9a7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:36.674124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7caa95-3ea6-4554-8e28-aa626ac39c6b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.675628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 42/74] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:36.677960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.678254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.678533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.696437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:36.697679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:36.712192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.712766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.713066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.713347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.714011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:36.714319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:36.715704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:36.723889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 33/75] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:36.724472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.747436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.765429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 43/76] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:36.766291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.833114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a2ec-43ba-46ad-83ae-cf9847316a37 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.866904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.867507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.867792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.868100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.868817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:36.869157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:36.870533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:36.878343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 34/77] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:36.881865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.894520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:36.903721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93d991-2bb3-4647-9406-e4d8c0492e99 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.939898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:36.940534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:36.940824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:36.941122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:36.942231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:36.943681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:36.949864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 44/78] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:36.949864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:36.991178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba4b2-ac95-46fc-9e5f-f68992e8ffbe None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:36.998234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.026218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.027056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.027488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.027932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.029252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.030872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.033191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 35/79] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:37.034099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.050348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8903-25ef-476e-bda1-5b55c87ee06a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.083922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.084562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.084850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.085159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.098886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.100364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.102114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 45/80] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:36 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:37.102954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.118157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.143668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbe963-3cde-4a26-a9b5-ac25a4a7d5d2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.176063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.176690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.176992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.177273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.178400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.179829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.181437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 36/81] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:37.185834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.210257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec8160-97ab-4cf7-839f-dfbaaef91a4a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.221681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.245644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.246182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.246461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.246734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.247374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.248826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.275497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 46/82] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:37 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:37.275497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.298367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f0f0c-3bdf-4efa-9584-983a0225b403 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.342695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.343358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.343675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.344568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.356984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:37.362480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:37.363551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:37.364429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 37/83] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:37.365081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.378202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.411711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560878e-c932-481a-a752-b1b329daf677 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.449783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.450523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.450811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.451694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.483101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c20d9-d54e-4e6b-a886-a4ab919661ce None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.492676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.533761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.534319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.534609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.534889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.538537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.540044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.619800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.621570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.631232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.661179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 47/84] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:37.662290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.735566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 38/85] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:37.736405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.757222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.805100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667ad0-1eb3-4ac9-9e10-2c3abecaf03b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.842063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.843035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.843538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.845196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.863151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a396-eda5-459c-b769-cfdeafa76e90 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:37.873289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:37.904863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:37.905836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:37.906297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:37.906837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:37.907789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:37.908299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.910143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:37.919342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 39/86] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:37.920541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:37.935137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:37.936941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.024531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.035367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82470d-e875-4572-9d7c-5991e3d0a278 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.072553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 48/87] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:38.073794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.089490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.090939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.091445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.091911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.093809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.096144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.098759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 40/88] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:37 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.103794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.148760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.193431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8598d3-a016-495f-bebc-5643013f0b6f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.229509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.230343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.230769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.231212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.232507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.234183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.235900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eec329-11f1-4184-a7b0-19cafc01d500 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.236668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 49/89] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.241857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.277737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.278168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.278871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.278871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.279601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.282125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.284845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.290553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 41/90] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.291339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.355877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605c06-891d-408f-a08e-9502ba71f049 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.391791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.392947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.392947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.392947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.394154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.395622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.397477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 50/91] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.402520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.439240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac693b4-cc60-48ec-a87a-ace4c3efe4b5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.442024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.486876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.487498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.487832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.488317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.488844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=3c3232bfea3b429697a9f4a4bfb3705a) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:38.489192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.490626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.501611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 42/92] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/role_assignments?user.id=3c3232bfea3b429697a9f4a4bfb3705a&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.501611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.535844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3338ba-9622-4e9e-9fd2-66b4bb618c6d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.537814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.612046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.613074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.613550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.614255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.615752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:38.616263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.621011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.633164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 51/93] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=5aa5cafea64a411885edceea9373b3df => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.643548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:38.688887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1ea-3291-4779-b6b9-83626537faf1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.762924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.763596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.763844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.764084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.832713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b691d76-10af-4d29-bca7-8ae9cbd83a05 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:38.863124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:38.917717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:38.918360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.918653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.918935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.919683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:38.920002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.921531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:38.930844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 52/94] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:38.946538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:38.948049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:39.112524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 43/95] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:39.118534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:39.174382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:39.272649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c1c9-ae46-4ae6-b1fa-3a672671fc49 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:39.319020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:39.319674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.319955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.320485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] PATH_INFO: `/v3/projects/5e1e9ada26824add8c5bdc9bcc691970` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.321639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] RBAC: Authorizing `identity:get_project(project_id=5e1e9ada26824add8c5bdc9bcc691970)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:39.323093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d67a6a-2858-4b4b-9c8b-45658589746c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:39.324899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 53/96] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:39 2022] GET /identity/v3/projects/5e1e9ada26824add8c5bdc9bcc691970 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:39.333666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:39.449409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:39.494406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6478b1f-3417-402e-8429-dddeca2d9f28 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:39.545795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:39.547377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.547884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.548353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.549376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=5e1e9ada26824add8c5bdc9bcc691970, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:39.549900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:39.551539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dd15c-02e0-41ed-9fbe-3c50ce6b7b5a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:39.561450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 44/97] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:39 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=5e1e9ada26824add8c5bdc9bcc691970&role.id=5aa5cafea64a411885edceea9373b3df => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:39.570893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f582c-8d30-4aff-9357-241a7766b761 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.571207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f582c-8d30-4aff-9357-241a7766b761 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.571459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f582c-8d30-4aff-9357-241a7766b761 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.572829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 54/98] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:39.578704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9351-c5b8-4568-9f6d-8724394f0090 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.579248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9351-c5b8-4568-9f6d-8724394f0090 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.579806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9351-c5b8-4568-9f6d-8724394f0090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.604426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c89f9351-c5b8-4568-9f6d-8724394f0090 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:39.606086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c89f9351-c5b8-4568-9f6d-8724394f0090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:39.688713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 45/99] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:39.919532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838cc9a-c2e5-41dc-a244-7f0e856dda7d None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.920000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838cc9a-c2e5-41dc-a244-7f0e856dda7d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.920000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838cc9a-c2e5-41dc-a244-7f0e856dda7d None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.921584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 55/100] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:39.926821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:39.927604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:39.928094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:39.948781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:39.950297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:40.024346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:40.035252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 46/101] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:40.249363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838cc9a-c2e5-41dc-a244-7f0e856dda7d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:40.378062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:40.385090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838cc9a-c2e5-41dc-a244-7f0e856dda7d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:40.432057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd0547-c433-441b-8413-09b1304130be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:40.432706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0547-c433-441b-8413-09b1304130be None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:40.433018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0547-c433-441b-8413-09b1304130be None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:40.433305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd0547-c433-441b-8413-09b1304130be None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:40.434433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd0547-c433-441b-8413-09b1304130be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:40.435917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd0547-c433-441b-8413-09b1304130be None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:40.437872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 56/102] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:40.443077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:40.565513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3ba10-7cf4-4e5a-8902-47bfd0a70089 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:40.599486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd0547-c433-441b-8413-09b1304130be None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:40.622436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:40.623440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:40.623744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:40.624031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:40.624699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:40.626865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:40.682418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 47/103] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:32:40 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:40.769435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd0547-c433-441b-8413-09b1304130be None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:40.813717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:40.814768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:40.815247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:40.815770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:40.832921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:40.838867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:40.839775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:40.840547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:40.842945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 57/104] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:40.849471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:40.903869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:40.971280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4ebec-7f97-4cd6-abd9-1e2ec5e51ed4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:41.012058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:41.012685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.012992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.013278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.013962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:41.014273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:41.016187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7b44-2703-4a5a-a232-305d8594b42e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:41.031135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 48/105] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:41.037126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:41.092840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:41.196345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efa285-b602-4888-ac54-cf48df2f1941 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:41.230055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fc09-cad8-4723-ab6d-2f4f770a3392 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.230495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fc09-cad8-4723-ab6d-2f4f770a3392 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.230702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fc09-cad8-4723-ab6d-2f4f770a3392 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.232200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 49/106] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:41.241018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.241018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.241018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.241281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:41.241281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.241281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.241502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] PATH_INFO: `/v3/users/07bdb33cf7ae44d6affed7f858ff88d9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.273166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] RBAC: Authorizing `identity:get_user(user_id=07bdb33cf7ae44d6affed7f858ff88d9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:41.275023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:41.276988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 58/107] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:41 2022] GET /identity/v3/users/07bdb33cf7ae44d6affed7f858ff88d9 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:41.282825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:41.284363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:41.289522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:41.389822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 50/108] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:41.428234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:41.448013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f075b-795b-4059-b777-cd5cc719c18a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:41.492728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:41.493372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.493655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.493931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.583247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:41.584741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:41.674275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:41.697930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 59/109] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:32:41.704597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:41.745631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:41.779530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:41.876626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed04dfe-ccc1-4a66-9d49-a6f6c33980ae None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:41.913510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefd7f-a3a6-4192-90c9-e6db48b1c22f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:41.941072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:41.941973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.942669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.942951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.944361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:41.947004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1fa6-f677-4441-8766-8d3668c02a24 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:41.948972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 51/110] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:41 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:41.950120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86d57-3624-432c-91fa-7efed92d32ac None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.950400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86d57-3624-432c-91fa-7efed92d32ac None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.950670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f86d57-3624-432c-91fa-7efed92d32ac None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.951220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:41.951848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 52/111] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:41.952238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.952704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.953718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.958088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f6e71-fd6a-4174-b3a5-96b4785d9d4c None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.958375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f6e71-fd6a-4174-b3a5-96b4785d9d4c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.958652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f6e71-fd6a-4174-b3a5-96b4785d9d4c None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:41.960041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 53/112] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:41.960596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f6e71-fd6a-4174-b3a5-96b4785d9d4c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:41.970696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-51b9f4ef-12d2-4b9b-b584-9c45c3f2b160 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:41.976093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:41.976946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:41.977745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:41.980220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 60/113] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:41.983071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886c24-1e68-482d-abda-8ae3a69f000e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:41.983548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886c24-1e68-482d-abda-8ae3a69f000e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:41.984001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886c24-1e68-482d-abda-8ae3a69f000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.003370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ee886c24-1e68-482d-abda-8ae3a69f000e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:42.005113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ee886c24-1e68-482d-abda-8ae3a69f000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:42.053249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:42.085946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f6e71-fd6a-4174-b3a5-96b4785d9d4c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:42.089526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:42.090350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 61/114] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:42.092453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eada0-6c27-4045-8512-5955cf5a6df5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.092748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eada0-6c27-4045-8512-5955cf5a6df5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.093164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684eada0-6c27-4045-8512-5955cf5a6df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.130565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-684eada0-6c27-4045-8512-5955cf5a6df5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:42.133579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-684eada0-6c27-4045-8512-5955cf5a6df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:42.153373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:42.154285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.154780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.155285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.159289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=07bdb33cf7ae44d6affed7f858ff88d9) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:42.159834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:42.161545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:42.173262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 54/115] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:41 2022] GET /identity/v3/role_assignments?user.id=07bdb33cf7ae44d6affed7f858ff88d9&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:42.175140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:42.313069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 62/116] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:42.365404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c805ff-68a7-4cfc-a497-fe36c6b07aa6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:42.380478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5bd46-85c1-4aaa-9eeb-a71c29a2defb None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.380774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5bd46-85c1-4aaa-9eeb-a71c29a2defb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.381073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5bd46-85c1-4aaa-9eeb-a71c29a2defb None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.382629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 63/117] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:42.387557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.387844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.388123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.407416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:42.408604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:42.426334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:42.427277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.427787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.428240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.429245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:42.429882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:42.431269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:42.448933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 55/118] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:42.455444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:42.502663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:42.522594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 64/119] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:42.529800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:42.595428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:42.648175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027cea5-5451-45da-872e-1388efe42d42 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:42.712715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:42.713618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.713903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.715430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.718185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:42.720218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:42.723401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933afda7-6032-474d-95b3-7e34a9de9b52 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:42.725427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 56/120] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:42.732206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:42.787749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:42.788408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.788693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.788989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.808279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:42.824575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:42.825516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:42.826303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:42.829701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 65/121] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:42.831268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:42.899474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:42.904057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c112-f738-4ef4-a198-dc133e34faa5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:42.956703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:42.957669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:42.958113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:42.958568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:42.960860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:42.963409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:42.966587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 57/122] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:42.974188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.005533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.017083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fe5e4-2359-401e-b090-4f3cfbf580d1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.054000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.054804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.054929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.055410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.067333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:43.078804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:43.079741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:43.081077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 66/123] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:43.081802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.122131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c1bc6-d0eb-4be2-95d3-8a536837e6d5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.160751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.161314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.161723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.162000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.162873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:43.163116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.164591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.174801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 58/124] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.175681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.209233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.219173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5679f9b-6bb7-4e4f-9ce7-5a08adbe2898 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.261526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.262350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.262654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.262978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.276208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:43.282851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:43.283769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 67/125] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:43.284940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.307762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.315143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc676dd-9575-41d1-ad46-f6527b0b59ba None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.352620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.353604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.354150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.354423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.356383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, user_id=a48ab2dbb7ac436d8fc857a199e4e839, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.358116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.375949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 59/126] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:43 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:43.381804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.466604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe6fe-f198-4111-9ecc-3c79ed7565dd None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.551294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.553050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.553532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.554879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.555764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:43.556269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.558012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.571843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 68/127] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.573504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.585741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.602697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740d561-9c54-4491-85b8-c32f6da1f134 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.640681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.641904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.642388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.642904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.644339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.646122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.648138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 60/128] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.649182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.668182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.716462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e417d-43d4-4443-92c7-1574d4d18b2b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.761614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.762157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.762443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.762721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.763950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] RBAC: Adding query filter params (name=member) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:43.764484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.766257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.777711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 69/129] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.781100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.787687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfe4d4-4fe6-4873-8130-f87bd2c69d1f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.823387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.824057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.824388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.824670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.825839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.827279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f427-fd33-49cc-9323-280364090692 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.829288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 61/130] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.830300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454631-7b27-4797-8044-fabda76f6d39 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.830584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454631-7b27-4797-8044-fabda76f6d39 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.830856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454631-7b27-4797-8044-fabda76f6d39 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.832216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 62/131] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:43.832790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454631-7b27-4797-8044-fabda76f6d39 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:43.859980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.897522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655678c-cbf8-45da-a597-59aef4c71b7f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.934225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.935276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.935757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.936230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.937680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.939405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65ac5-6aea-4db1-82fa-4603c13df1dd None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.941609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 70/132] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.941856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454631-7b27-4797-8044-fabda76f6d39 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:43.943202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.943729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.944285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.958978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:43.965695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:43.967231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:43.989589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:43.990234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:43.990514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:43.990788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:43.992283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:43.993574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:43.995187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 63/133] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:43.995960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.043047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 71/134] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:44.044365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.127603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d15a64-745a-4706-ae25-7a652e7300df None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.147859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f1c48-153e-40f6-9370-80990ec2b006 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.149210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:44.186079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.187253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.187659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.188229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/3c3232bfea3b429697a9f4a4bfb3705a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.198707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, user_id=3c3232bfea3b429697a9f4a4bfb3705a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.200173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.206711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:44.208006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.209172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.209460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.209781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.212286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.214182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.216208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 64/135] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:43 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:44.217159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.229206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 72/136] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:44 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/3c3232bfea3b429697a9f4a4bfb3705a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:44.230002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.436542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc34d7d-cee2-4703-86e6-ad0e6be00582 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.444883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfd628-a7d5-476e-86ef-86c494b326d5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.485049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.485641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.486036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.486316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] PATH_INFO: `/v3/projects/5e1e9ada26824add8c5bdc9bcc691970` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.486477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.487155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.487407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.487504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] RBAC: Authorizing `identity:get_project(project_id=5e1e9ada26824add8c5bdc9bcc691970)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.487681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.489117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.489883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, user_id=a48ab2dbb7ac436d8fc857a199e4e839, role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.491236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 65/137] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:44 2022] GET /identity/v3/projects/5e1e9ada26824add8c5bdc9bcc691970 => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:44.491410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5de75-1674-4274-8c18-4c9506c9003f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.492002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.511583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 73/138] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:44 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:44.512947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4991d-7e3f-46e0-9709-0adb8b052ef1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.513258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4991d-7e3f-46e0-9709-0adb8b052ef1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.513541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4991d-7e3f-46e0-9709-0adb8b052ef1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.515140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 74/139] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:44.516585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4991d-7e3f-46e0-9709-0adb8b052ef1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.537674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:44.608586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:44.659284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87171-f0e9-4197-974c-d385595ded5a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.661935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4991d-7e3f-46e0-9709-0adb8b052ef1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.702780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.703525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.703810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.704082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] PATH_INFO: `/v3/projects/5e1e9ada26824add8c5bdc9bcc691970/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.706126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1e9ada26824add8c5bdc9bcc691970, user_id=a48ab2dbb7ac436d8fc857a199e4e839, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.707781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.714182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.714930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.715257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.715566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.716818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.718450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4061132-3c5c-4d8e-a5d3-1b7991dd90cf None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:44.720947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 66/140] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:44.722152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.722441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.722731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.724364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 75/141] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:44 2022] PUT /identity/v3/projects/5e1e9ada26824add8c5bdc9bcc691970/users/a48ab2dbb7ac436d8fc857a199e4e839/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:44.731779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:44.745885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:44.747085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:44.888295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 67/142] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:44.928378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396078e4-8828-4e15-afc6-9187c6d14856 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:44.952175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:44.977545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:44.978095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:44.978377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:44.978654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:44.979310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:44.980770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:45.009826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 76/143] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:44 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:45.266382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:45.303158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:45.396519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438729-4ca8-4091-88e0-d5d4d980da09 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:45.435579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:45.436282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.436574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.436852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.447949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:45.452833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:45.454259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:45.455034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 68/144] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:45.461765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:45.583971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493009e-8742-4fcb-a1d2-e23b0e80047d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:45.625817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:45.626442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.626727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.627002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.627861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:45.628174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:45.629774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:45.646150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 77/145] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:45.652115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:45.752601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:45.770590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138a50f-3fe4-41da-a37a-5b230e1ec6f7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:45.806353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:45.807032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.807331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.807606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] PATH_INFO: `/v3/users/07bdb33cf7ae44d6affed7f858ff88d9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.808724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] RBAC: Authorizing `identity:get_user(user_id=07bdb33cf7ae44d6affed7f858ff88d9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:45.810263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:45.812208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 69/146] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:45 2022] GET /identity/v3/users/07bdb33cf7ae44d6affed7f858ff88d9 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:45.817938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:45.861392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:45.927122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b55ad-b9db-47c7-b38b-cd3179e67899 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:45.965813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:45.966513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.966798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.967071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.968202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:45.969735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718a336-f320-4164-b2d2-ed973dda649f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:45.971783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 78/147] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:45 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:45.977141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:46.083069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df5-9333-4007-8bfd-8b3fae4c0677 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:46.120254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:46.120851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.121159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.121438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/07bdb33cf7ae44d6affed7f858ff88d9/roles/7cf6e6a827d34416bf970d7c8626f3e6` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.123452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, user_id=07bdb33cf7ae44d6affed7f858ff88d9, role_id=7cf6e6a827d34416bf970d7c8626f3e6)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:46.125470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcefa1c-abd1-42bb-9a5e-c333df7bee7d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:46.141860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 70/148] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:45 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/users/07bdb33cf7ae44d6affed7f858ff88d9/roles/7cf6e6a827d34416bf970d7c8626f3e6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:32:46.606423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853102f-317d-472d-b77e-baa52bedc3ff None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.606929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853102f-317d-472d-b77e-baa52bedc3ff None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.606999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853102f-317d-472d-b77e-baa52bedc3ff None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.608696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 79/149] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:46.615287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.615746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.615930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.642269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:46.643484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:46.647256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e6fc6-c685-480c-b176-be6bca9cfabd None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.647540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e6fc6-c685-480c-b176-be6bca9cfabd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.647816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e6fc6-c685-480c-b176-be6bca9cfabd None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.649199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 80/150] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:46.655185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.655749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.656272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.658088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:46.693952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:46.695733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:46.733343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 71/151] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:46.821718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 81/152] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:47.119582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.132381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.190604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.241535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207afdf-4a5a-43f2-a0b5-b3726e43b9c4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.279038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.279820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.280118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.280452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.292837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9ad684ed-758c-4623-8448-bc22267f891d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:47.299074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:47.299987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:47.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 72/153] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:47.301406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65fcce-a545-42f5-8f17-d0bb1653aebf None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.302208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65fcce-a545-42f5-8f17-d0bb1653aebf None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.302208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65fcce-a545-42f5-8f17-d0bb1653aebf None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.308503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 73/154] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:47.310099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65fcce-a545-42f5-8f17-d0bb1653aebf None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.319086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.371207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68425bbe-6e8d-4d85-84f0-626a05e075d0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.443805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65fcce-a545-42f5-8f17-d0bb1653aebf None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.458432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.459382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.459856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.460319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.474070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-651c6ef6-ae5f-409f-8408-64934466ddb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:47.484979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:47.485833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:47.486741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:47.487400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.487400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.487400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.487400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.487644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:47.487644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:47.487644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:47.489999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 82/155] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:47.491411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.491883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.492361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.500545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 74/156] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:47.506284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.530048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:47.531647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:47.594135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.622637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 83/157] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:47.624170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.632019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946eaa-7435-4ad2-a40a-5a3283046ded None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.656984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.672978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.672978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.672978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.672978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.673357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:47.673357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:47.673357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:47.681832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 75/158] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:47.695757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.753759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc455c2b-690b-4977-9d41-52bc6a2a47bf None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.793569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.794464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.794965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.795427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] PATH_INFO: `/v3/users/3c3232bfea3b429697a9f4a4bfb3705a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.797287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=3c3232bfea3b429697a9f4a4bfb3705a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:47.798954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:47.800723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 84/159] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/users/3c3232bfea3b429697a9f4a4bfb3705a => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:47.817807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.864414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5315f-e0bd-4afa-b7b4-223231aaaedd None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:47.910043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.924788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:47.929170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:47.929170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:47.929170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:47.929170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:47.929170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:47.930350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 76/160] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:47.953811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:47.965300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:47.986548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192ada-12c1-4207-9729-517f3f16e6e3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.068019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.068645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.071930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.071930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.140353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da6905-7f48-4af2-906c-ff1a323b83f5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.194653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.195273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.195813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.196092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.222054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:48.223647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:48.285673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:48.296592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:48.296592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:48.401200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:48.425102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 85/161] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:32:48.426356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:48.457691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 77/162] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:48.458531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:48.610775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed238cb0-cf10-4b1d-8a9a-28d38832fa0f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.660107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.660754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.661048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.661319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.673344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:48.678341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:48.683893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:48.684939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:48.685788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:48.686534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccea89a-dd73-4eae-97a1-b14ea0ee5ba5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.689351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 86/163] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:32:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:48.690333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:48.755533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.756597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.757108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.757531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.761628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:48.763226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24b8e-d9f0-4a98-a3af-95c96c878497 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:48.765037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 78/164] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:48 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:48.766360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a6e-c761-4ee9-bec2-ae462bf1d9d5 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.766798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a6e-c761-4ee9-bec2-ae462bf1d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.767216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a6e-c761-4ee9-bec2-ae462bf1d9d5 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.768789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 79/165] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:48.773899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c95a6e-c761-4ee9-bec2-ae462bf1d9d5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:48.796508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:48.867672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670c403-71f2-40ee-8162-5d08af8d5362 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.922580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.923224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.923500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.923775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.924899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:48.926391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc4595-c5a9-44c2-ab39-9162c12b77ec None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:48.928329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 87/166] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:48 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:48.929481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.929759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.930028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.944430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c95a6e-c761-4ee9-bec2-ae462bf1d9d5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:48.965983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:48.967191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:48.997661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:48.998283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.998566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.998837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.999505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:48.999805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.001746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.010142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 80/167] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:32:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:49.010886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.066946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 88/168] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:49.067814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.086725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:49.145401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:49.181222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83321ad-fb3b-41c3-b855-c9f639229b19 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.218768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.219392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.219669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.219940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.220675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:49.221002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.222499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.238593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 89/169] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=5aa5cafea64a411885edceea9373b3df => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:49.239937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.262995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf6c6-7fec-4e0f-a12b-39934ff0cad2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.301384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.302007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.302285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.302631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.303431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=3c3232bfea3b429697a9f4a4bfb3705a) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:49.303738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.305322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.311670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 81/170] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/role_assignments?user.id=3c3232bfea3b429697a9f4a4bfb3705a&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:49.426018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3c81b-64cf-4853-b1a4-44ca2ec79a6a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.453023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:49.476558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.477609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.477886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.478154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.479382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.481102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.482956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 90/171] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:49.490060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.617640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:49.620282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9330f-2ba0-4618-a2f3-5f7bc90282be None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.655772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.657299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.657765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.658206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.730057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.732275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.825327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:49.837260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 82/172] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:49.840214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.869980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.969866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8054c64-f309-4ad8-9e64-4e7dab5073e1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.988605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6f71-6487-4815-ba8f-2c9c3d6585e5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.026189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.026822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.027102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.027419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.037677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.038331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.038611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.038883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.038993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:50.040060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:50.041547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513090f-f31d-475b-a283-2e92d8033169 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:50.043476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 91/173] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-8acdeafc-dcd2-4f72-b179-dfd12ae5ebfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:50.043963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:50.044829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:50.045712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:50.047467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3d90-c6dc-455e-a7f0-6a2a9b99a945 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.047467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3d90-c6dc-455e-a7f0-6a2a9b99a945 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.047467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf3d90-c6dc-455e-a7f0-6a2a9b99a945 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.047467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 92/174] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:50.049217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 83/175] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:50.051498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3d90-c6dc-455e-a7f0-6a2a9b99a945 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.053180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f03e07-41ca-4156-84c6-561567899690 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.053464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f03e07-41ca-4156-84c6-561567899690 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.053739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f03e07-41ca-4156-84c6-561567899690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.073334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-35f03e07-41ca-4156-84c6-561567899690 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:50.075711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-35f03e07-41ca-4156-84c6-561567899690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:50.125925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.145986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.156979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 84/176] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:50.156979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f03e07-41ca-4156-84c6-561567899690 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.223097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf3d90-c6dc-455e-a7f0-6a2a9b99a945 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.254827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f03e07-41ca-4156-84c6-561567899690 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.268640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.269729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.270131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.270536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.271260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:50.271809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:50.273659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae9e-c18a-43ef-84ec-3a13e24533ae None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:50.286840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 93/177] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:50.289306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d602e6-1874-4a4f-86ea-a67b59533b16 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.289657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d602e6-1874-4a4f-86ea-a67b59533b16 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.289934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d602e6-1874-4a4f-86ea-a67b59533b16 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.293484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 94/178] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:50.296625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57330b6-7bcc-438e-9e37-90c5a5a1b72b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.298351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57330b6-7bcc-438e-9e37-90c5a5a1b72b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.298642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57330b6-7bcc-438e-9e37-90c5a5a1b72b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.300527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 95/179] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:50.301772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181db68-3f17-404f-b857-14bfd6efb8bf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.302095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181db68-3f17-404f-b857-14bfd6efb8bf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.303013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181db68-3f17-404f-b857-14bfd6efb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.307884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.308522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.308800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.309198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.309902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:50.310206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:50.313007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8025e75-3aa8-4ffa-a3d3-0dee14c507ed None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:50.332587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 85/180] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:50.333994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.334279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.334549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.343128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1181db68-3f17-404f-b857-14bfd6efb8bf None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:50.345473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1181db68-3f17-404f-b857-14bfd6efb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:50.368630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:50.369862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:50.409589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.444494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 86/181] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:50.445826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.469125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 96/182] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:50.470668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b7d6-e819-4966-af11-794ad1c879b4 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.470953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b7d6-e819-4966-af11-794ad1c879b4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.471230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b7d6-e819-4966-af11-794ad1c879b4 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.472794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 97/183] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:50.478446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.478822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.479089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.499133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:50.500983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:50.541775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.556873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21277673-54e1-414f-9e49-769b083f04e4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.585204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 98/184] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:50.586349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.593842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.594475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.594753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.595024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] PATH_INFO: `/v3/users/a48ab2dbb7ac436d8fc857a199e4e839` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.596153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] RBAC: Authorizing `identity:get_user(user_id=a48ab2dbb7ac436d8fc857a199e4e839)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:50.597737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:50.599611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 87/185] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/users/a48ab2dbb7ac436d8fc857a199e4e839 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:50.605610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.691849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.747851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138db12-4704-4ecc-ad9c-36d16ec748d3 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.803049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9810d15-d371-443a-8f1b-7eb69a469232 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.810564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.811214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.811490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.811759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.826349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:50.833825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:50.834728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:50.835509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:50.836577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 99/186] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:50.839671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:50.853454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.854074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.854411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.854688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.872177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.930360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:50.931807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:50.941209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5929c43-85bd-4c60-8a67-8506e7485e06 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:50.972722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:50.975343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:50.975977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:50.976256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:50.976529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:50.986809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:50.991707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:50.992650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:50.993474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 100/187] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:50.994111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.039225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 88/188] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:32:51.040109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.089724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.098926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeebc-7a7f-4613-81c8-281afb171aec None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.132354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.133011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.133306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.133579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.144097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:51.145501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba2be-6396-4fa0-a14a-da5c108f7de0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:51.149405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:51.150253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:51.151058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 101/189] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:51.151715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.180518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.181167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.181444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.181714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.182369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:51.182668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.184017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.192346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 89/190] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.193146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.195799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.253965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924484d0-5bff-4e54-9abb-d3b54d6ec8cd None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.286766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.287400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.287683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.287985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.289124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.290585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.292145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 102/191] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.292764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a37079-a9f0-4523-bb66-cf3755f4daa5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.293144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.316312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.326988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.327616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.327899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.328717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.329410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:51.329714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.331078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.338733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 90/192] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.339501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.395552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd1e66-1378-43fa-bdff-53bf72ad32c0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.400981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.428414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.429074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.429353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.429624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] PATH_INFO: `/v3/projects/5e1e9ada26824add8c5bdc9bcc691970` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.430717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] RBAC: Authorizing `identity:get_project(project_id=5e1e9ada26824add8c5bdc9bcc691970)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.432156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.433794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 103/193] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/projects/5e1e9ada26824add8c5bdc9bcc691970 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.434548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.443102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960acc22-0cba-4e38-a728-34c013fbd91d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.478122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.478724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.479003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.479272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.479933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:51.480230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.481639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.490618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 91/194] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.492004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.526387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.539034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7ae0-c24e-43a4-9c7a-c31fc50e287a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.572731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.573393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.573680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.573957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.586920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.588365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.590020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4f9c9-3d66-4698-afa6-382106eaf11a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.590309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 104/195] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.591116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.620592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.630318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.630927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.631210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.631485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.632418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.633584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.654961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 92/196] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:51 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:51.655615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.701730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8753b5-507f-409a-a3ec-ac388825ca8a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.744871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.745558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.745836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.746108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.747278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.748768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.756357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 105/197] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.756357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.774130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.774441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c655b8e-ab7e-4bc8-be99-c8ce81413b32 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.819080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.819782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.820027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.820307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.821768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=5e1e9ada26824add8c5bdc9bcc691970, user.id=a48ab2dbb7ac436d8fc857a199e4e839) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:51.821768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.823454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.829864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 93/198] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/role_assignments?user.id=a48ab2dbb7ac436d8fc857a199e4e839&scope.project.id=5e1e9ada26824add8c5bdc9bcc691970&role.id=5aa5cafea64a411885edceea9373b3df => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.830679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.882553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f16f-7c83-4216-b2f9-08507131516d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.895111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.931430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.932109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.932385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.932651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] PATH_INFO: `/v3/users/07bdb33cf7ae44d6affed7f858ff88d9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.934029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] RBAC: Authorizing `identity:get_user(user_id=07bdb33cf7ae44d6affed7f858ff88d9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.935521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:51.938368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 106/199] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/users/07bdb33cf7ae44d6affed7f858ff88d9 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.940053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.976651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda36e1-9d05-44ab-a155-c3a64d7d29de None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.024118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.024930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.025302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.025690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.054755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.088326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c92b0-7bb0-427b-ac4e-5969e2e9240b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.117908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.119382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.133769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.134540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.134830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.135107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.170657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.217306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.218928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.229089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 94/200] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:52.230132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.311445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.331209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a1b60-4d3e-40b1-ba84-b8f703f15023 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.335251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 107/201] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:52.336222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.372151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.373693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.374044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.374392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.396853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.458173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.459880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.466277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2e230-5f38-43f0-88bd-aeaf330c3ddf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.513132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.513772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.514053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.514428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.515822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.517332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.519449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 108/202] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.520228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.573894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.580204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 95/203] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:52.581271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.631351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba622-7ec8-410e-b696-4a1dcba14516 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.640817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.669708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.670442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.670773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.671144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.673021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.674585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.676647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 109/204] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.677773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.711529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f58dee-2014-4485-8983-f6a658b3c34a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.758276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.758886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.759167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.759443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.760173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=180c7ca215ab4fe9b885322f01fad849) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:52.760484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.762502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.770334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 96/205] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/role_assignments?user.id=180c7ca215ab4fe9b885322f01fad849&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.771151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.792488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a3adf-263b-4993-9183-3f98319560fe None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.818834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.842987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.844544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.845125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.845410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.846739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.848827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.851037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 110/206] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.862087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.897943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284dd6e-fb74-4dd0-b3f9-e77d55e66a48 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.910362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.939916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.940537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.940878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.941186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.941978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=180c7ca215ab4fe9b885322f01fad849) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:52.942297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.943879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617820ce-67a6-438b-8781-dda0ae6e1423 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.951464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 97/207] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/role_assignments?user.id=180c7ca215ab4fe9b885322f01fad849&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.996371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c10344-2c28-427b-8444-09a358fed3ad None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:53.039486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:53.040115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:53.040392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:53.040662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:53.041422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, scope.project.id=cfa47402b0b8483c88bda7a3649db115, user.id=07bdb33cf7ae44d6affed7f858ff88d9) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:53.041729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:53.043264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481c72f-f776-484e-9b6a-99a5ed5a8d07 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:53.052179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 111/208] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/role_assignments?user.id=07bdb33cf7ae44d6affed7f858ff88d9&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:53.943340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce77bc-4b41-4518-92d0-11e01a9c32b3 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:53.944042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce77bc-4b41-4518-92d0-11e01a9c32b3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:53.944042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce77bc-4b41-4518-92d0-11e01a9c32b3 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:53.945801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 98/209] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:53.951017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:53.951307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:53.951588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:53.970963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:53.972136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:54.025172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:54.040425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae1837-29e7-4cb8-902a-d5603823eabd None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.041028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae1837-29e7-4cb8-902a-d5603823eabd None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.041538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae1837-29e7-4cb8-902a-d5603823eabd None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.043452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 99/210] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:54.050879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.051518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.051996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.052529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 112/211] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:54.071545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:54.073097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:54.147772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 100/212] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:54.322195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:54.421944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa075a4-e952-4450-8b33-a2f1e16b9dca None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:54.426798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:54.463421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:54.464058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.464331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.464600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.465726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:54.467221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:54.469401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 113/213] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:54.474718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:54.529829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:54.560370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b2e58-62fd-4d20-b6e0-3cc50be6f7a7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:54.564143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:54.592093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1288d-3baa-4f34-805b-15d1a02b670c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:54.609294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:54.611196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.611713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.612436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.626619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:54.631743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:54.632350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.632625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.632894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.633563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:54.634618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:54.635479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:54.636287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:54.636882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:54.638666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 101/214] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:54.646361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:54.665052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 114/215] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:54 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:54.771590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13322f2-b120-420b-b439-dc3c4db077cf None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:54.809880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:54.811089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.811616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.812088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.813083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:54.813595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:54.815363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e7f53-1ee9-4469-981a-4402ac18af9f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:54.825510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 102/216] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:54.831963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:54.907018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b0e5a-684f-487f-979a-0b714b461a9b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.907306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b0e5a-684f-487f-979a-0b714b461a9b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.907501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b0e5a-684f-487f-979a-0b714b461a9b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.909001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 103/217] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:54.914163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d2deb-b32e-47f0-8d43-08ac28c5c773 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.914454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d2deb-b32e-47f0-8d43-08ac28c5c773 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.914733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d2deb-b32e-47f0-8d43-08ac28c5c773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.934500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-977d2deb-b32e-47f0-8d43-08ac28c5c773 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:54.935702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-977d2deb-b32e-47f0-8d43-08ac28c5c773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:54.943311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71f144-e81c-4f04-a1ec-bb3fa448cbf0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:54.946504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:54.965108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:54.990815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:54.991463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.991746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.992025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] PATH_INFO: `/v3/users/5ca78713171f43c9b723f36f9d637823` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.007568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca78713171f43c9b723f36f9d637823)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.009129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.010792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 115/218] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:54 2022] GET /identity/v3/users/5ca78713171f43c9b723f36f9d637823 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:55.017217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.028172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 104/219] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:55.114735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047fa2-afa3-4cb7-b0d5-0807a76dba25 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.115027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047fa2-afa3-4cb7-b0d5-0807a76dba25 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.115309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f047fa2-afa3-4cb7-b0d5-0807a76dba25 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.116668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 105/220] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:55.121600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.121895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.122174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.138599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56164e-7917-4c57-95fe-496677f896f8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.143536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:55.144731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:55.179535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.180160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.180443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.180720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.214284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.237055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 106/221] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:55.262217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.263987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.335463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.371505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 116/222] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:32:55.386510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.436844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.456674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dfe0f-5f57-4fe0-acce-b9c99a597b4a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.555400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.555400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.555400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.555400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.569780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:55.573377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:55.580614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:55.581553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:55.582341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:55.585018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 107/223] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:55.610837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.641060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6525c-0a26-45af-91ae-5edb17348aba None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.726613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.729659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.730663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.731223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.733489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.739787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.745199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 117/224] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:55.748902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.784140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c6a38-80ee-430f-a0ea-d5c2ece3b958 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.826363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.827168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.827562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.828036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.829669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:55.830127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.831799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.841992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 108/225] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:55.845330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.867873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.954341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fa8f-76b9-4178-8c04-ba264f422128 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.018676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea4daf-a6d9-421a-bc51-24063dba65a4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.034637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.043464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.044119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.044400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.044726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.060048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:56.060683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.061536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.062006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.062498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.063433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=5ca78713171f43c9b723f36f9d637823) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:56.063946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:56.065100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:56.065977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:56.066757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 118/226] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:56.067872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.068619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.077238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 109/227] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/role_assignments?user.id=5ca78713171f43c9b723f36f9d637823&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=5aa5cafea64a411885edceea9373b3df => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:56.078982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.223472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.259570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e598-01b1-4a01-a7f4-5e83f1117890 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.291013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbfcc0-19f9-4d18-94e1-821c2bd0bda0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.325230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.325895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.326174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.326443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.329344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.330823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.341100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 119/228] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:56.342469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.389671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.390301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.391156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.391438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.392114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:56.392420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.393838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.415227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 110/229] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:56.416023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.420730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.464780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09777f2c-488f-40a3-8ac5-23376fbb0c2d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.508383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.509426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.509898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.511964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.514771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.519375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.523413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 120/230] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:56.529269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.589085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af8307-6df2-4f2d-b693-0d4636e2c5fc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.619319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.619959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.620238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.620512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.621690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.623118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.630152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 111/231] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:56.630985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.633872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.750480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fa9ac-4147-4b54-bec6-2743b757730f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.755646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a799-5fab-40ee-85a5-e447c430394b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.787601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.788250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.788531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.788805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.790846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.792290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.796526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 112/232] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:56.804680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.817285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.834046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.834719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.835002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.835321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/users/180c7ca215ab4fe9b885322f01fad849/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.838434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, user_id=180c7ca215ab4fe9b885322f01fad849, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.839962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb3fdc-6fe3-404f-87a8-33d6cf9703f1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.881472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 121/233] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:56 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/users/180c7ca215ab4fe9b885322f01fad849/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:32:56.913161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.968091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62cd2-0716-4d8b-8c28-ae4e2fe6ba74 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.009791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.010522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.010810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.011089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/users/180c7ca215ab4fe9b885322f01fad849/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.013309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, user_id=180c7ca215ab4fe9b885322f01fad849, role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.014903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f61c83-6039-46a5-9c8c-f25fe81e6e32 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.039161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 113/234] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:56 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/users/180c7ca215ab4fe9b885322f01fad849/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:32:57.113997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca127a8-cd3d-47f4-a0bd-46158f6ba986 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.114282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca127a8-cd3d-47f4-a0bd-46158f6ba986 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.114464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca127a8-cd3d-47f4-a0bd-46158f6ba986 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.115956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 122/235] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:57.131022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.131346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.131589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.154534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:57.155932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:57.216971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc460e7-1a78-4a8e-91bf-8dd01210ff9e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.217274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc460e7-1a78-4a8e-91bf-8dd01210ff9e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.217580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc460e7-1a78-4a8e-91bf-8dd01210ff9e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.219468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 123/236] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:57.225689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dac16-4b19-4661-9893-a276413345cf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.225972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dac16-4b19-4661-9893-a276413345cf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.226241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dac16-4b19-4661-9893-a276413345cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.246751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 114/237] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:57.253882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d03dac16-4b19-4661-9893-a276413345cf None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:57.255080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d03dac16-4b19-4661-9893-a276413345cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:57.332999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 124/238] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:57.548040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.631829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.681635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2f69-ce77-4e72-9f6e-865bcd554702 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.735087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.735727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.736003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.736271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.749375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:57.754778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:57.755690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:32:57.756492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 115/239] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:57.761708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dac16-4b19-4661-9893-a276413345cf None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.800281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.918985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cab75c-9e48-4e40-9763-5d7010a8121a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.932032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dac16-4b19-4661-9893-a276413345cf None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.940419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.957137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.958107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.958589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.959087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.960412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.962123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.964196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 116/240] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:57.974415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.975171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.007961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.008578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.008862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.009163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.009834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:58.010136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.011531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.024945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 125/241] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:58.030814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.134228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee8df-4576-43ad-84d1-9981c41cb709 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.170529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.171138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.171424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.171732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.172394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.173854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.199940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 117/242] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:58.223076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95031057-1383-465d-8936-1c95ca259c7d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.278385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.279044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.279327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.279605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] PATH_INFO: `/v3/users/234db021da034bbea0192271efa01926` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.303543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] RBAC: Authorizing `identity:get_user(user_id=234db021da034bbea0192271efa01926)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.305234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fb6ef-211d-42fa-bbe5-66672bc13dcb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.307298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 126/243] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:58 2022] GET /identity/v3/users/234db021da034bbea0192271efa01926 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:58.312789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.389523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:58.449931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e83a0-27b3-4209-87e7-0fc0b60fdf48 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.485165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.485808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.486088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.486362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.557133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.558979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.571497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7132-3170-42b1-81e7-f437c47618eb None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.571999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7132-3170-42b1-81e7-f437c47618eb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.572073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c7132-3170-42b1-81e7-f437c47618eb None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.573648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 127/244] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:58.578707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.578991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.579269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.600472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:58.603120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:58.617238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:58.639880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:58.699082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 128/245] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:58.700725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 118/246] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:58.706278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.822422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce217e6-cc9b-4aa0-8aed-5897ad63923c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.857939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.858583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.858863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.859262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.860321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.861795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.863741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 129/247] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:58 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:58.869192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.992128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa77b-2aae-47a5-b746-2c6e6e5f03b0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:59.010275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.039019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:59.040035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.040498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.040974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.041990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=234db021da034bbea0192271efa01926) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:59.042481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:59.044236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:59.050194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:59.052212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 119/248] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:32:58 2022] GET /identity/v3/role_assignments?user.id=234db021da034bbea0192271efa01926&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:59.139779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a225d6-cf52-4e44-bc11-7cf2b3a9d173 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:59.178293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:59.178947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.179282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.179562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.192304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-b25cc1a4-2513-4f20-aa60-b073cd3e2d38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:59.197697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:59.198571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:32:59.199374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:32:59.202153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 130/249] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:59.207918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.224616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:59.313429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99b148-f1c4-478b-820f-9b1a20231da2 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.313735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99b148-f1c4-478b-820f-9b1a20231da2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.313916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99b148-f1c4-478b-820f-9b1a20231da2 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.316319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 131/250] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:59.321546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fedbbd-cdf5-4a97-b7bd-4f1b0f1675ec None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.321829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fedbbd-cdf5-4a97-b7bd-4f1b0f1675ec None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.322672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fedbbd-cdf5-4a97-b7bd-4f1b0f1675ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.348326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03439c-2254-4823-9e40-94b7fa42a367 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:59.350887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e6fedbbd-cdf5-4a97-b7bd-4f1b0f1675ec None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:59.352090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e6fedbbd-cdf5-4a97-b7bd-4f1b0f1675ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:59.386941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:59.387959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.388501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.388993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.389865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:59.390446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:59.392287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:59.401848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 120/251] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:32:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:59.409939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.472778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 132/252] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:59.474140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d177501-9bb7-47c4-ba3b-7fe08f23c972 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.474432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d177501-9bb7-47c4-ba3b-7fe08f23c972 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.474711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d177501-9bb7-47c4-ba3b-7fe08f23c972 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.475962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 133/253] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:59.482286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.482570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.482851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.510552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:59.516731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:59.517964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:59.555756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9b141-7d40-42b1-83ef-7131ea1756b3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:59.572105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:59.621836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:59.623459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.623751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.624034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] PATH_INFO: `/v3/users/5ca78713171f43c9b723f36f9d637823` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.626367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca78713171f43c9b723f36f9d637823)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:59.628479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:59.630076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 121/254] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:59 2022] GET /identity/v3/users/5ca78713171f43c9b723f36f9d637823 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:59.636394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.684149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 134/255] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:59.784668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365ee57-1cd7-4d26-b338-05306d6aab36 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:59.833449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:59.834436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.834921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.835394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.841681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:59.843391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:59.846191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 122/256] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:32:59 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:59.852141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.882836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:59.898393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.028782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de584f-6608-4260-9d58-ef5b46a38872 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.110824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc088d5c-7b7c-496a-bf4b-4afcaf1bfaad None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.115923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.116825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.117533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.118049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/users/5ca78713171f43c9b723f36f9d637823/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.124326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, user_id=5ca78713171f43c9b723f36f9d637823, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.126550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.148014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 135/257] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:32:59 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/users/5ca78713171f43c9b723f36f9d637823/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:00.150594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.177412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.184575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.185249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.185530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.185803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.204095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:00.211083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:00.212422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:00.213281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 123/258] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:00.218635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.300011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0bfff-e257-4522-8329-7ef8495d87f4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.340475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.341476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.341902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.342351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.346062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.354911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:00.361031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:00.361959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:00.362776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:00.365653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 136/259] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:00.374178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.410603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d6fbc-6954-4a49-bd3f-e167b553d8ee None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.460089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.461795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.462095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.462374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.463244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:00.463562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.465097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.476830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 124/260] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:00.481448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.491465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.541802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728130d-019c-45d0-bb32-d9d1c76ba03b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.630879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.631477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.631717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.631948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.632613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:00.632918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.634335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.666546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 137/261] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:00.676036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.731870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ec9e-6648-4be0-8252-07a923e0208e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.785937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.809603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.810652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.811146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.811598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.817248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.818942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.821041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 125/262] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:00.826879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.847218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfaba4-1a21-454e-aaa7-ca772312f6fc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.882796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.883409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.883675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.883954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] PATH_INFO: `/v3/users/180c7ca215ab4fe9b885322f01fad849` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.887077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] RBAC: Authorizing `identity:get_user(user_id=180c7ca215ab4fe9b885322f01fad849)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.887077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.888647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 138/263] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/users/180c7ca215ab4fe9b885322f01fad849 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:00.895054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.904288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.043644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf9d0-6567-4463-a435-90bc6200a7f0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.107496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775235c0-d831-4714-b007-aa89069724d6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.118603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.119229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.119512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.119785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.161832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.162786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.163286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.163812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.188592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.196984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.198833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.292009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.293948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.298784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.302721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 126/264] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:01.308478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:01.405815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.413496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0950-d559-4773-8fcb-be22c8ca9f25 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.450107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.450747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.451027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.451303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.452668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.454707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.461066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 127/265] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:01 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:01.462476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:01.467688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 139/266] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:01.473785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:01.588657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6dc6d-08d9-46c4-9628-62d38416ff08 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.596993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd58fad-1bb9-4181-a478-704168705405 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.599440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.630151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.631403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.632098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.632783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.634415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=180c7ca215ab4fe9b885322f01fad849) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:01.634931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.636531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266297f-e0e2-4947-8b04-6de1c5d04b0b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.647244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 128/267] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:01 2022] GET /identity/v3/role_assignments?user.id=180c7ca215ab4fe9b885322f01fad849&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:01.648821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2992-1bcd-4dc8-b738-7af4fb2c8313 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.649130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2992-1bcd-4dc8-b738-7af4fb2c8313 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.649415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2992-1bcd-4dc8-b738-7af4fb2c8313 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.650964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 129/268] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:01.658892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.659566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.659838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.660643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.661283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.662805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.664877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57191d-f517-4ec8-98c1-ece95f1212b9 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.665185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57191d-f517-4ec8-98c1-ece95f1212b9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.665462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57191d-f517-4ec8-98c1-ece95f1212b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.669950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 140/269] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:01 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:01.678086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:01.716325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-7a57191d-f517-4ec8-98c1-ece95f1212b9 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:01.719099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7a57191d-f517-4ec8-98c1-ece95f1212b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:01.774276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.828932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 130/270] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:01.872763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64abcf-82c7-4ef8-b71c-59d19df33c4a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.922087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c0915-59c7-4e0e-b618-1f8ddf668e2f None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.922697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c0915-59c7-4e0e-b618-1f8ddf668e2f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.923183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c0915-59c7-4e0e-b618-1f8ddf668e2f None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.927635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 131/271] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:01.931718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.932006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.932223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.932349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.932849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.933155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.933432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.934167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=180c7ca215ab4fe9b885322f01fad849) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:01.934477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.935919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3725-49ec-4bbf-8ff3-7e1c33f9a87f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.943303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 141/272] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:01 2022] GET /identity/v3/role_assignments?user.id=180c7ca215ab4fe9b885322f01fad849&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:01.966149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:01.967385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:02.073987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 132/273] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:02.228325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3cdaa-16c4-458f-9cc9-e52c722fe896 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.228683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3cdaa-16c4-458f-9cc9-e52c722fe896 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.228882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3cdaa-16c4-458f-9cc9-e52c722fe896 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.230667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 142/274] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:02.235849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.236133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.236409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.237189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.270307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:02.271496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:02.304044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.347115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 143/275] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:02.378255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.395381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca2c2b-c1d0-4984-beeb-f0137ba613ac None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.406607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.434293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.435371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.435893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.436365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.448931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:02.454907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:02.455795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:02.456590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:02.459216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 133/276] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:02.467765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.523710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13d28c-5fb9-4158-8aab-961815941670 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.560120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.560754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.561341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.561341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.562989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:02.564468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:02.566404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 144/277] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:02.574569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.589450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c98911-96a7-4449-a349-d21a4f1e9b67 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.617305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.630225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.631157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.631764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.632237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.633158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:02.633665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:02.635392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:02.645056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 134/278] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:02.646096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.660201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.693132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cef799-2d03-4e96-bd90-22dc5502e59a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.729321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.729934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.730216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.730487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.731397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:02.732571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:02.761524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 145/279] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:02 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:02.761524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.765626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48a950-1a38-4946-9564-6e6425459f52 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.813566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.814191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.814481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.814759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.831176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:02.838596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:02.839773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:02.840751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 135/280] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:02.846491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:02.890523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.901265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bb569-da59-46f6-8964-c950176d471b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:02.945115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:02.946080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.946371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.946640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] PATH_INFO: `/v3/users/baed8be64125410584792c748f3cef42` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.960500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:02.964139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] RBAC: Authorizing `identity:get_user(user_id=baed8be64125410584792c748f3cef42)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:02.965813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edff6e0-03aa-48fc-ac4e-984ddd616e35 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:02.967656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 146/281] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/users/baed8be64125410584792c748f3cef42 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:02.974771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f710836-f397-4d48-95af-93dcb51f07b8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:02.975208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f710836-f397-4d48-95af-93dcb51f07b8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:02.975491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f710836-f397-4d48-95af-93dcb51f07b8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:02.976822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 147/282] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:02.977741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f710836-f397-4d48-95af-93dcb51f07b8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.002128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083d72e-ff81-49b8-b613-7f83cc2fe4d2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.044694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.045436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.045721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.045999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.047245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:03.047543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.048918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c79817-2599-422d-aab9-4e1fd14271d0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.059931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 136/283] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:03.061228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.061517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.061794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.087579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:03.089161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:03.115830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f710836-f397-4d48-95af-93dcb51f07b8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.153702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.154376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.154630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.154902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.155668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, user.id=baed8be64125410584792c748f3cef42, scope.system=all) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:03.156671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.158204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.164094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 148/284] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:02 2022] GET /identity/v3/role_assignments?user.id=baed8be64125410584792c748f3cef42&scope.system=all&role.id=5aa5cafea64a411885edceea9373b3df => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:03.164935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.181709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:03.184181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 137/285] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:03.286212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5bfcb-5f75-4037-9113-a49cbbffa9aa None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.324706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.325390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.325671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.325942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] PATH_INFO: `/v3/users/234db021da034bbea0192271efa01926` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.327097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] RBAC: Authorizing `identity:get_user(user_id=234db021da034bbea0192271efa01926)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.328697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.330415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 149/286] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:03 2022] GET /identity/v3/users/234db021da034bbea0192271efa01926 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:03.335996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.406006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:03.452107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb54d7a-dccc-4b5f-9432-b7202c6e5b97 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.492875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.493780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.494243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.494675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.501741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.503271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.505312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 138/287] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:03 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:03.510935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.567250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.623851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:03.664020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aad9d-bd49-4e99-9b3c-b4ef02813d05 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.725373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.726118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.726336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.726589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/users/234db021da034bbea0192271efa01926/roles/7cf6e6a827d34416bf970d7c8626f3e6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.734950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, user_id=234db021da034bbea0192271efa01926, role_id=7cf6e6a827d34416bf970d7c8626f3e6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.736444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.749677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7f691-37f2-482a-a79d-8a7b335149af None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.771990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 150/288] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:03 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/users/234db021da034bbea0192271efa01926/roles/7cf6e6a827d34416bf970d7c8626f3e6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:03.784304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:03.800163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.801202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.801628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.802981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.820098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:03.825832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:03.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:03.827955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:03.830382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 139/289] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:03.836078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.982929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e3731-d262-495a-9edb-bc2db4ab31c7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.024987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.025845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.026269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.026704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.027508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:04.027951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.029597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.039275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 151/290] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:04.044721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:04.171788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596416ee-c481-4d8f-884e-ddb5ac4a49d6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.219600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.220241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.220523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.220800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] PATH_INFO: `/v3/users/5ca78713171f43c9b723f36f9d637823` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.221981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca78713171f43c9b723f36f9d637823)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.223422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.225336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 140/291] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:04 2022] GET /identity/v3/users/5ca78713171f43c9b723f36f9d637823 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:04.231000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:04.237810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:04.367625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd52aa-ba0f-4e74-847d-2e8fc45d54f8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.400973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.401884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.402368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.402818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.474965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.476442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.478425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:04.616554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 152/292] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:04.623416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:04.699240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:04.744453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffba6a-2ae6-4c2a-9e6f-b1b051fdd9a6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.778720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.779352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.779640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.779916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.781604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.783042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c339b-f15c-4d7b-bf14-e45c2aab5d48 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.784669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 141/293] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:04 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:04.790647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:04.904872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf037f5-c0c7-4a9a-9cf2-934bbcdfd8b9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.906988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:04.954632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.955243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.955524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.955800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.956522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=5ca78713171f43c9b723f36f9d637823) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:04.956830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.958314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a272c-baa5-43b5-8b50-5cf94b0ede1e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.964525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 153/294] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:04 2022] GET /identity/v3/role_assignments?user.id=5ca78713171f43c9b723f36f9d637823&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=5aa5cafea64a411885edceea9373b3df => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:05.396656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766acdb-74f2-4c87-9183-a522af683c0b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.397066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766acdb-74f2-4c87-9183-a522af683c0b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.397245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766acdb-74f2-4c87-9183-a522af683c0b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.398830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 142/295] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:05.404109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f2ad-c453-4195-a76f-c16c42863c4c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.404405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f2ad-c453-4195-a76f-c16c42863c4c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.404689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f2ad-c453-4195-a76f-c16c42863c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.425155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6318f2ad-c453-4195-a76f-c16c42863c4c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:05.426358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6318f2ad-c453-4195-a76f-c16c42863c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:05.475852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f45-c991-4b81-b643-aa0fd9237074 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.476155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f45-c991-4b81-b643-aa0fd9237074 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.476393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f45-c991-4b81-b643-aa0fd9237074 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.478808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 143/296] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:05.483162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.483444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.483724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.503110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 154/297] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:05.509501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:05.509850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:05.544866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee04ef-0b88-433b-b474-dcced45a5492 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.545558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee04ef-0b88-433b-b474-dcced45a5492 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.545991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee04ef-0b88-433b-b474-dcced45a5492 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.548355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 155/298] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:05.554196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.554595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.554974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.582209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:05.583565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:05.587000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 144/299] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:05.658846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 156/300] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:05.800050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:05.858282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:05.876932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:05.911866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae149d0-4a1e-4dc5-a239-6cf645b9c56a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:05.913183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:05.963410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:05.964057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:05.964339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:05.964859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:05.966570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:05.968280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:05.976155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 145/301] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:05.976155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.003572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dc617-139f-4400-8a0e-a6c93ff59a22 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.040008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.040646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.040935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.041245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.051665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:06.056530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:06.057463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 157/302] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:06.059207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.085975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d6d7a-8106-40d7-8abd-3d241fadbbb5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.110631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.124421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.125621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.126339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.126861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.136971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:06.141871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:06.142801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:06.143624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 146/303] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:06.144287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.158804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.174045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639380c-3e22-4424-babf-6a289ac62904 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.208635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.209266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.209555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.209831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] PATH_INFO: `/v3/groups` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.210449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.211887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb65f3-af7a-4ce8-9a9b-a9f4e98fde3f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.226567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 158/304] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Aug 31 09:33:06 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:06.227754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dca47-8cb2-44f2-a0a6-cb5637d4c5c1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.228041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dca47-8cb2-44f2-a0a6-cb5637d4c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.228318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dca47-8cb2-44f2-a0a6-cb5637d4c5c1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.229568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 159/305] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:06.230364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dca47-8cb2-44f2-a0a6-cb5637d4c5c1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.261616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f8724-b0aa-4341-8445-2a18af2985f9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.301270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.301890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.302168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.302439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.303112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:06.303414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.304828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829bffd-5afc-4ce4-9163-c89c33ea0664 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.314692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 147/306] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:06.316242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.316532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.316805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.339044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:06.340891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:06.370991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dca47-8cb2-44f2-a0a6-cb5637d4c5c1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.393776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.410718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.411335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.411616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.411890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.412552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:06.412857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.414568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.425080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 160/308] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:06.425235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 148/308] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:06.425328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.431300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.435450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.539219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ae705-a789-4db4-9c33-663ceac47ba0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.557743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ddac5-0b10-42b0-91a4-fd83fb99a0be None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.578449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.579314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.579760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.580250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] PATH_INFO: `/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.594698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.595399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.595570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1334a83b6d4a2e9496fc0ad438a50b)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.595678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.595941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] PATH_INFO: `/v3/users/baed8be64125410584792c748f3cef42` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.597103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=baed8be64125410584792c748f3cef42)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.597804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.598535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.599770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 161/309] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:06.610639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 149/310] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/users/baed8be64125410584792c748f3cef42 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:06.610801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.612331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.679638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.692767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.735291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5db64-6554-437a-98d4-7c4be45e0656 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.743783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34e3d9-0885-4a53-9d78-21eed7212ff6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.770669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.771289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.771566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.771838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.772576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, user.id=ca1334a83b6d4a2e9496fc0ad438a50b, scope.system=all) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:06.772856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.774324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.778696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.779511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 162/311] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/role_assignments?user.id=ca1334a83b6d4a2e9496fc0ad438a50b&scope.system=all&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:06.779634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.780188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:06.789402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.789782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] PATH_INFO: `/v3/system/users/baed8be64125410584792c748f3cef42/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.802124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=baed8be64125410584792c748f3cef42, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:06.803548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:06.821015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 150/312] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Aug 31 09:33:06 2022] PUT /identity/v3/system/users/baed8be64125410584792c748f3cef42/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:06.930355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899fb4f-d7e9-4d50-b067-1547870642a8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:06.967519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:06.974723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:06.975368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:06.975647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:06.975923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:06.986550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:06.991489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:06.992588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:06.993544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 163/313] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:06.998384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.106863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb19bf4-17ca-4b6e-ac3a-7a0ab1180b0b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:07.117294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.144184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:07.144806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.145130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.145414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.146092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:07.146402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:07.147781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:07.157519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 151/314] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:07.163121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.267432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7816e8a-e39e-4dfe-8679-4ad6ae7bd115 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:07.304850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:07.305803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.306259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.306705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] PATH_INFO: `/v3/users/234db021da034bbea0192271efa01926` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.308026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] RBAC: Authorizing `identity:get_user(user_id=234db021da034bbea0192271efa01926)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:07.309704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:07.311862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 164/315] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/users/234db021da034bbea0192271efa01926 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:07.317324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.426255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36ced5-2ad8-4b4f-86e1-d6f7a23bac97 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:07.461533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:07.462145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.462431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.462708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.503626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.533497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:07.534868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:07.635833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 152/316] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:07.641577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.705727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.747492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3134c-4afe-4f5d-998c-c7604a5acc25 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:07.782492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:07.783127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.783405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.783733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.784799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:07.786279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c87d8e-4561-45f5-9c55-39045b39b94f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:07.788213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 165/317] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:07.793991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.871989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.907109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11e8c-40ca-4066-9739-4aa1bca61ade None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:07.945414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:07.946638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.947112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.947575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.948487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, scope.project.id=38c47769a3424022a8b4840633730b81, user.id=234db021da034bbea0192271efa01926) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:07.949058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:07.950690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e53f0f-1c7f-4a5d-9262-5cc4b712e1e7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:07.957666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 153/318] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/role_assignments?user.id=234db021da034bbea0192271efa01926&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.196754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cc7d-355c-4a43-9be2-a867c1fb2ea1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.197286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cc7d-355c-4a43-9be2-a867c1fb2ea1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.197453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cc7d-355c-4a43-9be2-a867c1fb2ea1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.198787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 166/319] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:08.203705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.203995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.204271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.222253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:08.223443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:08.293736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 154/320] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:08.567695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324cc7d-355c-4a43-9be2-a867c1fb2ea1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.668031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324cc7d-355c-4a43-9be2-a867c1fb2ea1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.693863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.703598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.704229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.704504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.704772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.705874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.707316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70217fe-f0b6-4476-b422-32f0caf1016c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.709068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 167/321] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.714425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.777581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.791052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f898b-21e5-4380-acb0-4d80aed74283 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.791359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f898b-21e5-4380-acb0-4d80aed74283 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.791616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f898b-21e5-4380-acb0-4d80aed74283 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.792984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 168/322] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:08.798334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.798623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.798905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.817185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:08.817843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837fda4-9b9a-478e-aabb-c59571cbf86f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.818382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:08.853583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.854185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.854462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.854734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] PATH_INFO: `/v3/groups` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.855348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.856762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7933c1e-9e3e-4fba-b3d4-0347be957c43 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.874618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 155/323] 10.222.0.25 () {60 vars in 1134 bytes} [Wed Aug 31 09:33:08 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:08.876434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0ae4-57a0-4f49-988c-bf47281a61bd None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.876724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0ae4-57a0-4f49-988c-bf47281a61bd None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.877019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a0ae4-57a0-4f49-988c-bf47281a61bd None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.878318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 156/324] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:08.884044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.884367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.884592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.891550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 169/325] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:08.904974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:08.906305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:08.981011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 157/326] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:09.164764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.261979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.270239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f8f2-aa7c-4f09-9bc8-c055b64e5120 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.299784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.314967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.315612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.316348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.316637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.328801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:09.332580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:09.336096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:09.337072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:09.337905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 170/327] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:09.344857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.397043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320227-4d31-488d-b837-106e5a4c5982 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.441775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.442426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.442701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.443042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.463927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:09.469377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad03c41-e256-49b5-995b-cfd98dacb9b4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:09.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:09.471233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:09.472026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:09.472840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 158/328] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:09.478581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.515880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.516772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.517088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.517368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.518191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:09.518552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.520018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.532167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 171/329] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.537458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.598997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.611735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fde26-d369-4fb3-a284-effb8ed1fceb None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.631837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.664168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.665355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.665355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.665355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.665797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:09.666103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.667513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.672019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198936-071a-4279-b819-5ea10e673029 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.678234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 159/330] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:09.684317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.717542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.719221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.719819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.720340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] PATH_INFO: `/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.721763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1334a83b6d4a2e9496fc0ad438a50b)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.723438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.726682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 172/331] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:09.733137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.806775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abfe27-57da-4121-b01f-63b7f3f2fade None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.844162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292c7e7-3d3d-4b09-9240-357576cb4885 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.862784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.866566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.867221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.867506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.867784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] PATH_INFO: `/v3/users/baed8be64125410584792c748f3cef42` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.868942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] RBAC: Authorizing `identity:get_user(user_id=baed8be64125410584792c748f3cef42)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.870504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.872458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 160/332] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/users/baed8be64125410584792c748f3cef42 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:09.878751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.884069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.885057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.885587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.886099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] PATH_INFO: `/v3/system/users/ca1334a83b6d4a2e9496fc0ad438a50b/roles/7cf6e6a827d34416bf970d7c8626f3e6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.887742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ca1334a83b6d4a2e9496fc0ad438a50b, role_id=7cf6e6a827d34416bf970d7c8626f3e6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.889555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5c9e0-9dfc-4d81-b9a9-4a405668a60a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.908469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 173/333] 10.222.0.25 () {58 vars in 1334 bytes} [Wed Aug 31 09:33:09 2022] PUT /identity/v3/system/users/ca1334a83b6d4a2e9496fc0ad438a50b/roles/7cf6e6a827d34416bf970d7c8626f3e6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:33:09.945031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.024238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd936f4-7f90-41ed-b06a-d39a578c4c6b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.070991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.072002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.072508 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.073013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.074070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] RBAC: Adding query filter params (role.id=5aa5cafea64a411885edceea9373b3df, user.id=baed8be64125410584792c748f3cef42, scope.system=all) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:10.074602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.076267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2bc66-e312-4b83-9670-86f45fa2d3d3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.096929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 161/334] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/role_assignments?user.id=baed8be64125410584792c748f3cef42&scope.system=all&role.id=5aa5cafea64a411885edceea9373b3df => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:11.553787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd7b9-c56c-437c-80f5-42d6fe4f63d5 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:11.554301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd7b9-c56c-437c-80f5-42d6fe4f63d5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:11.554376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd7b9-c56c-437c-80f5-42d6fe4f63d5 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:11.555882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 174/335] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:11.561218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219fbfd-9073-4d5a-b6e8-763245e11ba3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:11.561661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219fbfd-9073-4d5a-b6e8-763245e11ba3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:11.561852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219fbfd-9073-4d5a-b6e8-763245e11ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:11.589946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e219fbfd-9073-4d5a-b6e8-763245e11ba3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:11.591147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e219fbfd-9073-4d5a-b6e8-763245e11ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:11.676061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 162/336] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:11.813255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b2033-7b4b-489c-add3-447f5af6c5a8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:11.813548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b2033-7b4b-489c-add3-447f5af6c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:11.813724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b2033-7b4b-489c-add3-447f5af6c5a8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:11.815296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 175/337] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:11.820394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa006d-19e0-4468-ab44-13803e20e428 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:11.820682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa006d-19e0-4468-ab44-13803e20e428 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:11.821234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa006d-19e0-4468-ab44-13803e20e428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:11.845961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-72fa006d-19e0-4468-ab44-13803e20e428 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:11.847152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-72fa006d-19e0-4468-ab44-13803e20e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:11.882604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:11.936565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 163/338] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:12.060640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc90e-92f2-4fca-a461-3cff9dde8445 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.061015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc90e-92f2-4fca-a461-3cff9dde8445 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.061348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc90e-92f2-4fca-a461-3cff9dde8445 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.064110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 176/339] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:12.075034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.075324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.075607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.101957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:12.103161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:12.191655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 164/340] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:12.227423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecc90e-92f2-4fca-a461-3cff9dde8445 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.347236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecc90e-92f2-4fca-a461-3cff9dde8445 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:12.388797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:12.389463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.389746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.390018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.409086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:12.414941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:12.415838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:12.416912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:12.423867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 177/341] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:12.430479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.446298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.491529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:12.589001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:12.592725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e17d-5c7e-42ce-aea6-1443f02e4126 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:12.594060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524ba8f-f027-40e7-9697-b240012954c0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:12.637728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:12.638472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.638831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.639172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.654571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:12.655496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.655958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.656372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.656482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:12.657322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:12.657764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:12.659314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:12.662508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:12.663594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:12.664610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:12.668988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 178/342] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:12.672786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.676925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 165/343] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:12.681352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.801940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:12.810158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd130d1d-3b91-4203-b85a-79b68173cc7c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:12.840793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e444708-c6b7-4c06-84b4-859edd15b897 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:12.848507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:12.849428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.849907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.850376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.851267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:12.852217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:12.853836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:12.865881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 179/344] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:12.867102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:12.938294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:12.938928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:12.939210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:12.939488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:12.982728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.016745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586f7a4-a17e-44b0-b5a8-f7e5c6cd553b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.034249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.035706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbdc36-9eb5-4828-ac35-924eca6cce3a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.054874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.055582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.055868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.056153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.067349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:13.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:13.073934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:13.074871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 180/345] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:13.075573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.122310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.180442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 166/346] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:13.182343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e18c0-c8a4-44e1-9c7d-f1296af22c94 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.182713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e18c0-c8a4-44e1-9c7d-f1296af22c94 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.183062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e18c0-c8a4-44e1-9c7d-f1296af22c94 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.185055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 167/347] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.190365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e18c0-c8a4-44e1-9c7d-f1296af22c94 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.259398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc96a-f34f-4e5e-bee2-c20ce6986da9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.267786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.300163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.301088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.301585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.302173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.328927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e18c0-c8a4-44e1-9c7d-f1296af22c94 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.371461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.372078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.372362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.372637 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.373350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:13.373780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.375833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8e9ac-4278-4d41-b77a-180de73e56a7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.385549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.386446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 168/348] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:13.387017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42012d3-c707-4fbd-a339-cafa06089644 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.387678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.387966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.388242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.410006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.411414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:13.412677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:13.495047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 169/349] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:13.496174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.528376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 181/350] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:13.532980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d28f-89ba-4595-a7bd-cf80a1c010a9 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.532980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d28f-89ba-4595-a7bd-cf80a1c010a9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.532980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5d28f-89ba-4595-a7bd-cf80a1c010a9 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.532980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 182/351] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.533836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5d28f-89ba-4595-a7bd-cf80a1c010a9 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.562068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.626110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a3cad-9088-47b1-948a-89e3a28b3604 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.668986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5d28f-89ba-4595-a7bd-cf80a1c010a9 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.671211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.673368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.673655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.673934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.675086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.676530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5871b-5247-4c28-b154-7eef85546a64 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.678680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 170/352] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:13.679835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.680120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.681549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.693325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.705680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:13.706915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:13.713193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.714000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.714281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.714552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.798955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 171/353] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:13.799726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.817105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.818733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d90ed-765f-48dc-b14f-c604c2fcf8bf None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.855582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:13.923407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953313-1fe1-48a1-b60d-3dd81c3558a3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:13.941660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 183/354] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:13.943210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03e2e-8e98-4857-b3d8-230ef2e3ae8a None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.943692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03e2e-8e98-4857-b3d8-230ef2e3ae8a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.944153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d03e2e-8e98-4857-b3d8-230ef2e3ae8a None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.946119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 184/355] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.947047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d03e2e-8e98-4857-b3d8-230ef2e3ae8a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.967363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:13.968007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.968357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.969304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.970855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:13.972166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:13.973819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 172/356] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:13.975193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:13.977262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.074812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d03e2e-8e98-4857-b3d8-230ef2e3ae8a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.103044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626ee44-fbdd-4ea3-92b6-0b7255d4f947 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.117208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.117528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.117841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.118115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.118444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.125070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.126592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.128201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 185/357] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.128976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.143034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.143686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.143959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.144235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.154896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-c671b88e-6ac1-4bdc-ac7d-0000d9045d06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:14.160413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:14.161390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:14.162182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 173/358] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:14.163114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.164458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.164741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.183483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:14.184065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:14.231036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.255129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699360-c48e-437b-be93-a85b9c3b7b74 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.264114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 174/359] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:14.265048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.296949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.297687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.297971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.298245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.299858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.301570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.307688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 186/360] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.308425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.378839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d580a-0909-49d3-9e0d-9852ac3d3e8c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.382673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.420420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.423896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.423896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.423896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.426268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.426544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.428378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 175/361] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.428979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.442311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec06e1-4d1a-4af4-a32b-99dbed95c804 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.489457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.490103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.490385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.490658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.503488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:14.508900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:14.509819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:14.510623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 187/362] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:14.511307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.526899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.553478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24f8cd-a620-41af-8b4a-6d406f1a42ff None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.600145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.600762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.601086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.601343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.602064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=cfa47402b0b8483c88bda7a3649db115) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:14.602369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.603804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.614690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 176/363] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=5aa5cafea64a411885edceea9373b3df => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.615510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.634292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a16dbc-998b-4713-a7dd-b42a59a688f3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.666913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.681056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.681707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.682052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.682333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.683125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:14.683440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.684876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.697443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 188/364] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.698343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.755789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb879ba-b257-40f8-aade-4e29d654f89a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.788872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.802595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.802948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.803232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.803501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.804609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.806074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.807797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 177/365] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.808469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.817884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6db5cd-7da0-4ac2-a4b4-1fd88ab64bdf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.865439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.865664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.865984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.866093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.866884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=cfa47402b0b8483c88bda7a3649db115) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:14.867109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.868635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.875192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 189/366] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.876021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.934005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40610f-827c-4a64-9699-66d807fecccc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.936537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.986197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.987198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.987692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.988158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.990636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:14.990636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.991556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.014337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 178/367] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.016220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.042002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd802f-6902-4057-a7cd-1ecd140b280a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.107423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.108075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.108361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.108636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.124258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.130512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:15.138214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:15.139143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:15.139920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 190/368] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:15.141223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.190804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da694967-edea-47ac-9010-a2949f81b90d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.231603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.232242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.232519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.232839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] PATH_INFO: `/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.234094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] RBAC: Authorizing `identity:get_user(user_id=ca1334a83b6d4a2e9496fc0ad438a50b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.235729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.237645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 179/369] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/users/ca1334a83b6d4a2e9496fc0ad438a50b => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.238323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.277441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7cf95-a6df-4790-a8f2-7a41a05fa1cc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.288065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.322629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.323442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.323724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.324132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.324915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=38c47769a3424022a8b4840633730b81) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.327129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.329901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.339467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 191/370] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=5aa5cafea64a411885edceea9373b3df => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.340467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.404566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1bb8-7cb8-4efb-9e24-fe19af13da0e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.462980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.463608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.464250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.464534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.464811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.493977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ba0e9-87e3-4eb7-93bb-09bebf258509 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.544678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.547077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.547077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.547077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.547077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.547442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.548757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.561801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 192/371] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.564569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.600033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.604412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.665345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.726266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d865de-68d3-4d91-b66d-f55508e5801a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.754959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.767030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.767812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.768323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.768663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.769667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] RBAC: Adding query filter params (role.id=7cf6e6a827d34416bf970d7c8626f3e6, user.id=ca1334a83b6d4a2e9496fc0ad438a50b, scope.system=all) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.769983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.771489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.776949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 180/372] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:15.777889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.793325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 193/373] 10.222.0.25 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/role_assignments?user.id=ca1334a83b6d4a2e9496fc0ad438a50b&scope.system=all&role.id=7cf6e6a827d34416bf970d7c8626f3e6 => generated 510 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.794322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.955206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa95dc-d505-4d67-939e-ffdd5d323c30 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.992411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.993057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.993339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.993607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.994720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.996164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.996492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56df22e-93e9-407a-8183-04aea5b54e1e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.001084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 194/374] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.004316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.035601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.040313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.040938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.041425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.041714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.101368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.122045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.123518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.129375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587dae-fa3c-48e9-8bbf-98abd789ffa4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.164974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.165673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.165955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.166227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.167321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.168772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.172262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 195/375] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:16.177110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.240599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 181/376] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:16.258109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.288893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f639-7e1d-4dc6-b7fe-abc59a7d8290 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.313437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.321384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.328220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.328886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.329196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.329466 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.330413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=38c47769a3424022a8b4840633730b81) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:16.330524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.332165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.344151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 196/377] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:16.438197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7355f0f-aa24-4b89-98cf-1451a95b5fae None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.491730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.492396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.492678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.492972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] PATH_INFO: `/v3/projects/908a4dee0a684e749e89f40e4287b799` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.494189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] RBAC: Authorizing `identity:get_project(project_id=908a4dee0a684e749e89f40e4287b799)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.495648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.497640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 182/378] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/projects/908a4dee0a684e749e89f40e4287b799 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.503920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.639966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7fb96-073e-4d51-9768-e8b5761c32a8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.678277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.678955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.679235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.679503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] PATH_INFO: `/v3/groups/94746d1181204135a2037533b99439b0` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.684501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] RBAC: Authorizing `identity:get_group(group_id=94746d1181204135a2037533b99439b0)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.686054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc22ee-b1c8-4ff6-ab9a-8cf90dcc3215 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.691677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 197/379] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/groups/94746d1181204135a2037533b99439b0 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.694457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.821394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.824064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6205-f6b8-482f-bba3-3df6f62e10eb None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.872008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.872617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.872893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.873185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.873907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] RBAC: Adding query filter params (group.id=94746d1181204135a2037533b99439b0, role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=908a4dee0a684e749e89f40e4287b799) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:16.874212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.876167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93e3dc-2204-4398-9697-502819fe38cc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.883115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 183/380] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/role_assignments?group.id=94746d1181204135a2037533b99439b0&scope.project.id=908a4dee0a684e749e89f40e4287b799&role.id=3058b83cad264d36bc1ba356d7561bef => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.906090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6eab-fc23-4342-852f-ed61fff0a7b1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.906565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6eab-fc23-4342-852f-ed61fff0a7b1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.906565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6eab-fc23-4342-852f-ed61fff0a7b1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.908012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 198/381] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:16.914077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278be4-208e-457f-a716-6731283d8b76 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.914369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278be4-208e-457f-a716-6731283d8b76 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.914642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278be4-208e-457f-a716-6731283d8b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.938611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-81278be4-208e-457f-a716-6731283d8b76 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:16.939813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-81278be4-208e-457f-a716-6731283d8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:17.031755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 184/382] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:17.367776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc6eab-fc23-4342-852f-ed61fff0a7b1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.429162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.431715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113050b-fda2-49d5-8f15-e4f6b4d049ed None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.432211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113050b-fda2-49d5-8f15-e4f6b4d049ed None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.432650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113050b-fda2-49d5-8f15-e4f6b4d049ed None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.434535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 185/383] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:17.439457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.439737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.440017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.460634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:17.461866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:17.529706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc6eab-fc23-4342-852f-ed61fff0a7b1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.582499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.583136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.583417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.583693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.584419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 186/384] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:17.602959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:17.608448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:17.609986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:17.610789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:17.614919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 199/385] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:17.616108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.642715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.726966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445473d-b540-4084-9975-57b98ba1bda7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.763108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.764085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.764575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.765104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.765990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:17.766491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.768042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.777170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 187/386] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:17.784991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.933505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0dce8-eefe-46f2-95df-da1dfceaed03 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.947193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.979089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.979724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.980001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.980269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.981390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.982823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.984694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 200/387] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:17.990211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.078884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.118831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:18.136119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b2f68-167f-4040-9121-fe6d09503bea None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.175089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.175737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.176021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.176298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.177489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.178995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.181274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 188/388] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:18.186343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.267798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e52a71-3f50-4527-a1c1-a500e0287d28 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.302855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.303494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.303772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.304042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.310373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007aa-a384-4a82-ae4e-a302a9e6b443 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.314403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8ce5fd5a-1397-4bf4-a53c-d37091d55a24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:18.320528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:18.321724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:18.323146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 201/389] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:18.324686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207433-8518-4218-a3f1-e4201311126e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.324686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207433-8518-4218-a3f1-e4201311126e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.324686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a207433-8518-4218-a3f1-e4201311126e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.326461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 202/390] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:18.327656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a207433-8518-4218-a3f1-e4201311126e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.357692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.358369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.358654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.358997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.361353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, group_id=caaefd05ae54401f9f08a6d54704c8aa, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.362793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78112a-b99c-43d4-bf73-70de7316ce6d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.364590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:18.389089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 189/391] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:18 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:18.390860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53507bf-94b7-4e74-b0a8-8284ce139d03 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.391396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53507bf-94b7-4e74-b0a8-8284ce139d03 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.391818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53507bf-94b7-4e74-b0a8-8284ce139d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.430082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c53507bf-94b7-4e74-b0a8-8284ce139d03 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:18.431552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c53507bf-94b7-4e74-b0a8-8284ce139d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:18.449069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a207433-8518-4218-a3f1-e4201311126e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.451688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:18.488785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.489704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.490031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.490305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.491513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:18.491812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.493539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.504453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 203/392] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:18.510637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.581384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 190/393] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:18.616030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b78d-f24a-46dd-9776-1daf1d561a05 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.616335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b78d-f24a-46dd-9776-1daf1d561a05 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.616610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b78d-f24a-46dd-9776-1daf1d561a05 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.618229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 191/394] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:18.626312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.626312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.626312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.646014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:18.647323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:18.662913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:18.669243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0516a-180d-48f5-b1eb-8932e213ba9c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.706614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.707208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.707496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.707773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.709002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.710427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.712374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 204/395] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:18.718030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.723855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 192/396] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:18.824724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21717f70-1762-4aa0-8815-487137ddb8e4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.860599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.861285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.861681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.861959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.863103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.864547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.866586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 205/397] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:18.872203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.929485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.976497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.004577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f0cc5-a9e4-4635-b487-be4068e952d1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.005306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.042188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.043213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.043691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.044202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.046682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa47402b0b8483c88bda7a3649db115, group_id=caaefd05ae54401f9f08a6d54704c8aa, role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.048318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.065029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 193/398] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:18 2022] PUT /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:19.095101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.177536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37dcb7-96a1-4135-b106-ac07247db0c6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.224632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b101f2-e572-4b7f-8c28-72688fb2b93e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.232319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.232894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.235577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.235869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.252409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:19.260523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:19.262369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:19.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 206/399] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:19.269785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.277445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.278279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.278709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.279115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.282603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.295714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:19.304309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:19.305268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:19.306058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:19.308616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 194/400] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:19.314734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.384749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.395401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff98c8-b98a-4991-b867-cbdc1c5b0b61 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.431918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.432883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.433674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.434375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.435717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] RBAC: Adding query filter params (name=member) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:19.436451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.438382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.439615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4066a6-7044-4c96-8816-6e73ad95f6e8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.449263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 207/401] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:19.455604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.484315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.485011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.485321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.485600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.486275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:19.486902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.488561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad920603-ec9a-44cf-b9ea-e9425fbb985d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.502481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 195/402] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:19.502481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60952980-019c-4c48-a81a-c580612515ee None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.502481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60952980-019c-4c48-a81a-c580612515ee None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.502481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60952980-019c-4c48-a81a-c580612515ee None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.502481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 196/403] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:19.506740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60952980-019c-4c48-a81a-c580612515ee None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.571911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46641e-add9-4a44-8df6-f9d7db058bfe None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.585185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.619814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.620812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.621235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.621897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.623537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.625169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5f035-189c-41e6-b96e-0c9bdbcbb69c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.632112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 208/404] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:19.632112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.632112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.632112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.646116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60952980-019c-4c48-a81a-c580612515ee None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.653181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:19.655038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:19.679448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.686469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.687388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.687867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.688288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.689789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.691423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.697055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 197/405] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:19.697055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.741186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 209/406] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:19.742238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.844913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690bcb-6b9b-4519-a187-8cd8abf03649 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.884581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.885515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.885962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.886425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.887779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.889386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.891679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 210/407] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:19.895977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596ec3-449c-42cc-931a-aed5d4b8cc89 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.897649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.920152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.922191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.933728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.934657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.935103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.935560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.936942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:19.938777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:19.940543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 198/408] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:19.948281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.009402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3682a-60c4-427d-8927-77334773b873 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.044488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.045168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.045447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.045717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.048514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, group_id=caaefd05ae54401f9f08a6d54704c8aa, role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.050097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.075150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 211/409] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:19 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:33:20.115088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e5fb-269d-4450-bd19-767c27459ed5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.155680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.156639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.157139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.157573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.159720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=38c47769a3424022a8b4840633730b81, group_id=caaefd05ae54401f9f08a6d54704c8aa, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.161756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.162804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.181014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 199/410] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:19 2022] PUT /identity/v3/projects/38c47769a3424022a8b4840633730b81/groups/caaefd05ae54401f9f08a6d54704c8aa/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:20.277234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.306936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfa4a-e2a8-4a2c-ab31-47763dc0670d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.349275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.350173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.350597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.351008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.363579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:20.369002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:20.370179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:20.371185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:20.373645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 212/411] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:20.380072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.439265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.493552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841d5a-baa4-402f-ad32-e11692acf1f4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.530337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.531704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.532129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.532561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.533401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:20.533863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.535386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.544923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 200/412] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.554113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.682636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577aba2-4ba6-495b-9707-3492a12e1251 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.720540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.721483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.721925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.722354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] PATH_INFO: `/v3/groups/94746d1181204135a2037533b99439b0` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.723613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] RBAC: Authorizing `identity:get_group(group_id=94746d1181204135a2037533b99439b0)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.725237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa4167-97a5-4e79-ae7c-a99ac5988232 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.727330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 213/413] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/groups/94746d1181204135a2037533b99439b0 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.738123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.837499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ff70-24d4-4c33-8c40-f94dabb54b7f None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.838107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ff70-24d4-4c33-8c40-f94dabb54b7f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.838595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ff70-24d4-4c33-8c40-f94dabb54b7f None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.840451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 214/414] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:20.845185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.845508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.845752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.862786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f5ce5-b8eb-4006-ae16-b87ea4fd5582 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.877115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.885965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:20.887175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:20.893268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.900021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.900662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.900939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.901237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] PATH_INFO: `/v3/projects/908a4dee0a684e749e89f40e4287b799` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.902375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=908a4dee0a684e749e89f40e4287b799)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.903805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.905786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 201/415] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/projects/908a4dee0a684e749e89f40e4287b799 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.911574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.015598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c03f2-968a-45d9-bf23-c9e508ff4ea8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.050577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.051239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.051516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.051790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] PATH_INFO: `/v3/projects/908a4dee0a684e749e89f40e4287b799/groups/94746d1181204135a2037533b99439b0/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.053692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] RBAC: Authorizing `identity:create_grant(project_id=908a4dee0a684e749e89f40e4287b799, group_id=94746d1181204135a2037533b99439b0, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.055116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.081017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 202/416] 10.222.0.25 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:20 2022] PUT /identity/v3/projects/908a4dee0a684e749e89f40e4287b799/groups/94746d1181204135a2037533b99439b0/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:33:21.081218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 215/417] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:21.474907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.523087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.604437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d9e0a-4f13-43b6-8a28-b5efcf702ada None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.638106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.638741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.639015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.639287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.650505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-279f9cc2-4b39-404e-a492-be0e4ffa63cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:21.656708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:21.657737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:21.659623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 203/418] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:21.664002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.694746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d437-e860-4c63-8d50-de3e47500634 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.695090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d437-e860-4c63-8d50-de3e47500634 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.695283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d437-e860-4c63-8d50-de3e47500634 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.697006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 204/419] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:21.702441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.703035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.703514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.722077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:21.723569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:21.812346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475aff-3c8d-4de8-b840-0262adf2f3ee None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.836397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 205/420] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:21.872899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.873982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.874588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.875174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.876273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] RBAC: Adding query filter params (name=member) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:21.877938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.882655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.888798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.894690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 216/421] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:21.902142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.935147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.026954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176e13d-d620-4097-b8cc-1a198369cc0f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.082049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.082596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.082879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.087828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.162017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.163481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.176462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.221331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.319820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 206/422] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:22.328997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.380868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403531af-a09a-4a17-83c6-808f26574792 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.438531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83d9f8-64f3-465f-acbb-d2c720ba38fe None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.450861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.451957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.453273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.453866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.469594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-780448a3-8a59-4562-b02b-fe0c89da3aad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:22.474885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:22.475777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:22.476592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 217/423] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:22.482131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5d59-25cb-4f70-b0f9-3c102224b81d None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.482497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5d59-25cb-4f70-b0f9-3c102224b81d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.482799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af5d59-25cb-4f70-b0f9-3c102224b81d None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.488692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 218/424] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:22.488692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af5d59-25cb-4f70-b0f9-3c102224b81d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.490704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.491473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.491853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.492226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.493669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.495204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecd8c6-7d5d-432c-90b4-cfbb8524c352 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.497281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.502106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 207/425] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:22.503424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.503709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.503986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.522839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:22.524059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:22.567577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.600982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 208/426] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:22.600982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.619821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af5d59-25cb-4f70-b0f9-3c102224b81d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.657031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.657569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.657852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.658186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.658965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:22.659272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.660692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb795ed-9d78-49af-9537-188f68d52e35 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.670533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 219/427] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:22.672339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4e3bf-2877-47af-82b4-ba16922788e0 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.672756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4e3bf-2877-47af-82b4-ba16922788e0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.673059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4e3bf-2877-47af-82b4-ba16922788e0 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.674674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 220/428] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:22.676818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4e3bf-2877-47af-82b4-ba16922788e0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.716801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a5a4-9ee1-4fb0-8f20-3a278d2ff7a9 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.748393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.756699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.757575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.758002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.758478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.759746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.761416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d017e5-caf2-40b8-a80c-2ce589286006 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.763546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 209/429] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:22.765072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.765554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.766017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.789218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:22.790684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:22.802903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4e3bf-2877-47af-82b4-ba16922788e0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.829123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.842570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.843973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.844557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.845050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.875528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 210/430] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:22.876073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.921901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.923728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.999376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9edf7-fd01-4def-8598-6e8f3d54742e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.027597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.033018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 221/431] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:23.034013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.038918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.039767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.040223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.040683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.041612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=cfa47402b0b8483c88bda7a3649db115) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.042096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.043718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.050356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 211/432] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=5aa5cafea64a411885edceea9373b3df => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.051415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.059824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.149425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67296063-1840-4d84-ba3d-dc926d5ef039 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.185980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.186658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.186923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.187194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.199006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:23.204830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:23.205828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:23.207042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 222/433] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:23.207740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.240989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea622b-534c-47dc-a55b-4a807b8f7b8e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.276729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.276729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.276729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.277099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] PATH_INFO: `/v3/projects/cfa47402b0b8483c88bda7a3649db115` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.278968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] RBAC: Authorizing `identity:get_project(project_id=cfa47402b0b8483c88bda7a3649db115)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.279713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.281789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 212/434] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/projects/cfa47402b0b8483c88bda7a3649db115 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.282384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.306147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.335668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bba1ba-1e46-4c9c-81be-3b4ffa9dd285 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.354560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.381420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.382219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.382506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.382782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.395584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:23.398107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:23.399141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:23.399944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 223/435] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:23.400600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.418759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a655-3da2-478f-ad9d-b3df2716539b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.464685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.465706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.466009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.466294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.466978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.467289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.469289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.478857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 213/436] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.480997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.528589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e082-8113-4f54-874c-389c5fa6529c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.582077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.583135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.583420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.583697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.584843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.586463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.588209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 224/437] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.589233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.601867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e5c-a1a9-4879-8e26-09e24b38ba47 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.602779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.631163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.643832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.644445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.644760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.645066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.645728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] RBAC: Adding query filter params (name=member) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.646036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.647393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.655369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 214/438] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.656176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.702059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23542ff-b6bf-4c02-8b35-fa6a1ace3c8f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.735619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.736153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.736438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.736715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.755233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1a7f-cde0-437c-9670-2e5d2469638d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.789788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.790388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.790676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.790953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.791672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=cfa47402b0b8483c88bda7a3649db115) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.791989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.793456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43882a38-bac1-4fc1-b2eb-daa759512308 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.799233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 215/439] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=cfa47402b0b8483c88bda7a3649db115&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.800352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ce329-f00f-4e6f-b22b-38afe2585e3a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.800638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ce329-f00f-4e6f-b22b-38afe2585e3a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.800923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ce329-f00f-4e6f-b22b-38afe2585e3a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.802444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 216/440] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:23.803193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ce329-f00f-4e6f-b22b-38afe2585e3a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.806427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.807958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d10a-d760-4b1b-b7e0-8e3bbb0c6de6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.820735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.885295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.923031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 225/441] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:23.924136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.924428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.924706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.944106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:23.945346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:23.955771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ce329-f00f-4e6f-b22b-38afe2585e3a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.992773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.993436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.993726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.994004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.034010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 226/442] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:24.034010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.086465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.087222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.089993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.150088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.169330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182ae12-4600-491a-9107-db50ecf89fb2 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.217572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 217/443] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:24.220026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.220671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.221029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.221259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.222609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.223660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.223871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.225554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 227/444] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.232346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.328515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370cef2-a085-44f2-88bf-d79824295ae3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.337618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50629425-7c04-4666-a51b-e8da6308851d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.347081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.362241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.362872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.363156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.363437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] PATH_INFO: `/v3/projects/38c47769a3424022a8b4840633730b81` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.364557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] RBAC: Authorizing `identity:get_project(project_id=38c47769a3424022a8b4840633730b81)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.366029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.372152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 218/445] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/projects/38c47769a3424022a8b4840633730b81 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.372152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.376367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.377025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.377309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.377610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.378732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.380169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.381822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 228/446] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:24.382567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.390598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.471741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d15c2e-2d57-4bee-b61a-d84c80ece9e2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.482715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d34b7-e9e8-4527-96b8-bd80c01aa717 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.505416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.506042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.506322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.506593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.517252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.517571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:24.517874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.518156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.518432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] PATH_INFO: `/v3/groups/caaefd05ae54401f9f08a6d54704c8aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.519535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] RBAC: Authorizing `identity:get_group(group_id=caaefd05ae54401f9f08a6d54704c8aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.521023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:24.522443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:24.528149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:24.529004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:24.529614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 229/447] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/groups/caaefd05ae54401f9f08a6d54704c8aa => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.529684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 219/448] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:24.529756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.529823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.568573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.606734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.623783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6611e1d-4b79-4199-9b19-19b8e3323cc6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.629828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9518b1-256a-464b-a5d6-1a718db3c1f0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.657100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.657696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.657982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.658255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.658972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=38c47769a3424022a8b4840633730b81) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:24.659282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.660715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.663255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.663854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.664132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.664404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.665145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] RBAC: Adding query filter params (group.id=caaefd05ae54401f9f08a6d54704c8aa, role.id=5aa5cafea64a411885edceea9373b3df, scope.project.id=38c47769a3424022a8b4840633730b81) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:24.665454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.666519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 230/449] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.666858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.667293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.688733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 220/450] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/role_assignments?group.id=caaefd05ae54401f9f08a6d54704c8aa&scope.project.id=38c47769a3424022a8b4840633730b81&role.id=5aa5cafea64a411885edceea9373b3df => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.794997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765b55a-51c4-4e9a-bcf1-b04ab08b0d75 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.806394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.838739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.839649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.839930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.840206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.840871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:24.841215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.842619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.852082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 231/451] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.858114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.962906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6021968-fd0a-4b71-bdd8-ae4d917f83cd None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.995983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.996598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.996883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.997191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.008375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.066038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.067483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.167074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 221/452] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:25.173651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.279058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfa1e6-8963-470c-b2ad-86e7aee2d775 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.312787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.313445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.313728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.314004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] PATH_INFO: `/v3/projects/908a4dee0a684e749e89f40e4287b799` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.315116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] RBAC: Authorizing `identity:get_project(project_id=908a4dee0a684e749e89f40e4287b799)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.316557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.318208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 232/453] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/projects/908a4dee0a684e749e89f40e4287b799 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.324055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.363449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.418192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa89ce-5aeb-4320-926a-91a8f9d90fe2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.451042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.451652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.451928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.452200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] PATH_INFO: `/v3/groups/94746d1181204135a2037533b99439b0` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.453393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] RBAC: Authorizing `identity:get_group(group_id=94746d1181204135a2037533b99439b0)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.454805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc315413-b275-4cf6-8d9e-a8083fa112f6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.456322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 222/454] 10.222.0.25 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/groups/94746d1181204135a2037533b99439b0 => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.461987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.501393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.559886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373d6a-1c35-4646-81ae-a992d6d37df1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.592569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.593725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.594007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.594279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.594997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] RBAC: Adding query filter params (group.id=94746d1181204135a2037533b99439b0, role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=908a4dee0a684e749e89f40e4287b799) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:25.595304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.596735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af517c8d-b55d-4aca-8562-afae2f35324d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.602551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 233/455] 10.222.0.25 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/role_assignments?group.id=94746d1181204135a2037533b99439b0&scope.project.id=908a4dee0a684e749e89f40e4287b799&role.id=3058b83cad264d36bc1ba356d7561bef => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:28.042172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921241f-d8a7-4ff8-ab91-68930edc73df None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.042709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921241f-d8a7-4ff8-ab91-68930edc73df None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.042709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921241f-d8a7-4ff8-ab91-68930edc73df None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.044354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 223/456] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:28.049923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c216b-fc36-431b-a99b-e98717433b4d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.050353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c216b-fc36-431b-a99b-e98717433b4d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.050472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c216b-fc36-431b-a99b-e98717433b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.080593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-377c216b-fc36-431b-a99b-e98717433b4d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.080593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-377c216b-fc36-431b-a99b-e98717433b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.090906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156096-d734-4889-978a-09b7aff84290 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.091192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156096-d734-4889-978a-09b7aff84290 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.091467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156096-d734-4889-978a-09b7aff84290 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.092845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 224/457] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:28.098283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a75bc-2001-4af9-a8c4-4900ca9bd962 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.098744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a75bc-2001-4af9-a8c4-4900ca9bd962 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.099168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a75bc-2001-4af9-a8c4-4900ca9bd962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.126337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e01a75bc-2001-4af9-a8c4-4900ca9bd962 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.127527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e01a75bc-2001-4af9-a8c4-4900ca9bd962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.210130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 234/458] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:28.236916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 225/459] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:28.377174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f13c-a1ec-4940-a577-7ceb33ddce3e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.377511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f13c-a1ec-4940-a577-7ceb33ddce3e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.377634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f13c-a1ec-4940-a577-7ceb33ddce3e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.379263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 235/460] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:28.384087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.384380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.384654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.406512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.407695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.445184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.492120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 226/461] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:28.525531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f13c-a1ec-4940-a577-7ceb33ddce3e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.571233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.620584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.690445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f13c-a1ec-4940-a577-7ceb33ddce3e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.746921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.747597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.747875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.748143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.763892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:28.765528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-d1a32be2-3e00-4d96-9e34-b26c3ffd9d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:28.783782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:28.784739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:28.785656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:28.785656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:28.785656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:28.786503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 236/462] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:28.788229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a696-d535-413c-a1f7-41715107263b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.788509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a696-d535-413c-a1f7-41715107263b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.788787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a696-d535-413c-a1f7-41715107263b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.792999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 237/463] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:28.795263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.795548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.795818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.797157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbddb-8102-4ef6-8c4d-e2c5813e55ee None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.816680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.817890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.833996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.834632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.834913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.835188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.846007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:28.847421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:28.863235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:28.864166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:28.865098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:28.865098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:28.865098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:28.865098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 227/464] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:28.865098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.897494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 238/465] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:28.897494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.900008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.917685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.006396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef3ccd-fdf1-4c2a-aab0-093dab71fb97 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.032773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a14f4-42a0-4bb8-b29a-178196968196 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.042369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.043553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.044026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.044469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.045309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.045804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.047882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.061002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 228/466] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.062217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.095060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.096474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.097153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.097566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.112133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.114274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:29.135883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:29.137029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:29.138079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:29.138079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:29.138079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:29.140494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 239/467] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:29.142008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.158110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.183369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea011-47e1-4708-be6f-4b6fa0d9f510 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.217053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.224791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.225461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.225744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.226028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.226697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.227007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.228366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.246008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 229/468] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.246008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.257963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65baebb-3d4a-49a6-92ea-0597e67a8499 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.292358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.292995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.293280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.293551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.294187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.295623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.324991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 240/469] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:29.324991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.376723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc64d73-4d93-48f4-bf4b-2250fadc4cd7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.425424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.426355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.426931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.427391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.428323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.428814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.430610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.440995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.442894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511af64f-8da9-4367-a4bf-e7a88bdc6da3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.445709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 230/470] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.447091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.471339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.485774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.486416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.486699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.486973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.499794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.501306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:29.516845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:29.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:29.518744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:29.518744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:29.518744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:29.518905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 241/471] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:29.519604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.566589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ead12-6127-40b9-b17c-cf4eeabbf72e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.607065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.607977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.608260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.608740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.609543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.611190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f353369-e4a0-4e66-aaa6-39717bc028ef None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.631402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729ab1f-4dff-4a05-aaa2-34e890f5305f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.634796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 231/472] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:29.635837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bebfb-1ce3-45e0-bffc-c651f36a2e0f None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.636123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bebfb-1ce3-45e0-bffc-c651f36a2e0f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.636394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bebfb-1ce3-45e0-bffc-c651f36a2e0f None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.642616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 232/473] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:29.642616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bebfb-1ce3-45e0-bffc-c651f36a2e0f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.679776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.680481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.680770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.681146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.681865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.683387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a539938-cbe2-49ec-b61f-e796fc2e45d3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.720247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 242/474] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:29.720247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.720247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.720247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.751723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:29.757993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:29.760579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.808815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.834424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bebfb-1ce3-45e0-bffc-c651f36a2e0f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.864988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 243/475] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:29.878513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.879231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.879510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.879837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.880491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.880790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.882263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.897576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 233/476] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.903358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.015513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d0fa-5001-4cc7-a136-20fa9b51e67f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.051903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.052508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.052783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.053078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.053730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.055162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.079183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 244/477] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:30.165992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.297643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2571ecb-a29f-4712-8391-b5b9c95f5793 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.318025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.337517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.338049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.338326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.338668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.339353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.340729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.354298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.sql.core [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:33:30.354298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:33:30.354298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [parameters: {'id': '21ee15908ee14ccb902a1284b82dcd01', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:33:30.354298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88241) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:33:30.371251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:33:30.372168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application yield resource Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise self.value Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application session.commit() Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:33:30.373164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:33:30.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:33:30.375546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:33:30.376382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [parameters: {'id': '21ee15908ee14ccb902a1284b82dcd01', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:30.378639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:33:30.380640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:30.386726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 234/478] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 31 09:33:30 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:30.389561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.410062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.497997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98f6f-f6e6-46a4-9119-dfcc77ebe0f9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.532771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.533763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.534259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.534687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.545691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-38623094-c5c3-4258-b4b3-7d358c38e88a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:30.551818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:30.552870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:30.553898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:30.556181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 245/479] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:30.561575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.700487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a04579-f81e-4921-ad84-ca33c4d680e0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.766218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.766845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.767125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.767396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.768067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:30.768373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.769749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bef01c-45bb-4288-8b05-87f8eda3053f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.785479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 235/480] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:31.505197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f2-065f-45ba-86cd-78b54c1ab65b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:31.505707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f2-065f-45ba-86cd-78b54c1ab65b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:31.505818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f2-065f-45ba-86cd-78b54c1ab65b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:31.507304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 246/481] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:31.512691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de22fb-bced-45ac-bdf7-b565211de3d8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:31.513110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de22fb-bced-45ac-bdf7-b565211de3d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:31.513274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de22fb-bced-45ac-bdf7-b565211de3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:31.541947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e0de22fb-bced-45ac-bdf7-b565211de3d8 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:31.543150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e0de22fb-bced-45ac-bdf7-b565211de3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:31.589633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:31.637437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 236/482] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:32.021282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918df6f2-065f-45ba-86cd-78b54c1ab65b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.095814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9679-f6fa-49f7-af45-8fae134ab396 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.098391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9679-f6fa-49f7-af45-8fae134ab396 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.098391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9679-f6fa-49f7-af45-8fae134ab396 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.098391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 237/483] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:32.103293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a1bc-ce33-4723-b845-a3290235fbd2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.103659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a1bc-ce33-4723-b845-a3290235fbd2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.104241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a1bc-ce33-4723-b845-a3290235fbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.137348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.141600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-da40a1bc-ce33-4723-b845-a3290235fbd2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.142787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-da40a1bc-ce33-4723-b845-a3290235fbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:32.164388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918df6f2-065f-45ba-86cd-78b54c1ab65b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.201111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.201983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.202959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.203380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.216267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:32.222578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:32.223681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:32.224686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:32.227438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 247/484] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:32.232784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.325301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 238/485] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:32.328695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381349e-5be2-4b0c-9f6c-8ba974df8968 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.331735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381349e-5be2-4b0c-9f6c-8ba974df8968 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.332168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381349e-5be2-4b0c-9f6c-8ba974df8968 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.337658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 239/486] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:32.338965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1381e-9769-4252-afda-6445d9d77d9e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.339411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1381e-9769-4252-afda-6445d9d77d9e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.339834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1381e-9769-4252-afda-6445d9d77d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.363714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5f183-829c-4153-aabc-7974a15c81a7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.380309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a3e1381e-9769-4252-afda-6445d9d77d9e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.382491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a3e1381e-9769-4252-afda-6445d9d77d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:32.429768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.430647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.430930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.431210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.432284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:32.432595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:32.440976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:32.442229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.452314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 248/487] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:32.462103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.488351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 240/488] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:32.618883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508e7a-9644-4a35-b8e9-d16683da10be None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.619459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508e7a-9644-4a35-b8e9-d16683da10be None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.619459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508e7a-9644-4a35-b8e9-d16683da10be None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.620999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 241/489] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:32.625772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.626172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.626451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.632282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cedaf1-6f1f-4c84-abc2-e31ae94bfff8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.656606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.657985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:32.680966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.681862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.682138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.682408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.686193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.696699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:32.699000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:32.716751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:32.718567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:32.719608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:32.719608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:32.719608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:32.720704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 249/490] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:32.721811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.735432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 242/491] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:32.736463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.812864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.870796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d9af3-fe7a-4382-82ad-b1d6cfee5b70 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.880511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2cf4-1e2e-401c-9b7c-01e5402ba91e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.907690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.908856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.909393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.909880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.916778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.917542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.917831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.918107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.918773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:32.919247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:32.920532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:32.922915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:32.929016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:32.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:32.930716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:32.937517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 243/492] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:32.937517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.938322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 250/493] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:32.939459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.962222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.042789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54cfaa-845e-48f5-946e-bf7942c7d252 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.049814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a8cd-0c84-4210-a38a-3f397757e228 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.061208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.082570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.083212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.083496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.083771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.089888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.090738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.090948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.091226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.096817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:33.100708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:33.102145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.103019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:33.103813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:33.104410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.104477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 244/494] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.104477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:33.121326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.122389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 251/495] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.123328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.200540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.220504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980095c3-2cc0-42e6-a5e7-68188ebf84c9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.251072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b4154-9cbc-4121-94d6-2353b27f51d8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.256343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.256980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.257266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.257543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.258215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.258521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.259885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.272564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 245/496] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.272564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.297270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.297901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.298188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.298462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.308661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:33.313995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.314883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:33.315696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:33.319775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 252/497] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.320556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.353268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.381925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc17e4-e5da-49be-88bf-6769b60f9887 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.419349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.420224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.420694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.421151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.421831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.422125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.423472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.429535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644a7a-9896-4c2b-9269-415dc6f21cc0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.438314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 246/498] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.438314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.449750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.466772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.467384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.467667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.468004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.468718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.469206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.470498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.513860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 253/499] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.514642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.552966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fda5-59f9-4229-8adc-7565faab1c43 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.586387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.587011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.587289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.587566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.598686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.600116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.600799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.613619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa963d5-1f98-4f40-8971-a23dc59f2327 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:33.614571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.615469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:33.616522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:33.616522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:33.616522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:33.616522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 247/500] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.616522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.650457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.650971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.651249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.651522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.652168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.652470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.653896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.661680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 254/501] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.662455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.672989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.716451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944d76c-140d-409e-94eb-dba9cfa86e31 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.750787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.751413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.751696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.751967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.762615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.764070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.767643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca72be5-a330-4f86-beaa-bc5436694645 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-a00ea11e-3c9f-42e5-a9df-42a8271d4160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:33.778900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.779835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:33.780695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:33.780695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:33.780695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:33.780695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 248/502] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.781204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88411-4f06-4724-a7a1-d7078a617626 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.781490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88411-4f06-4724-a7a1-d7078a617626 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.781723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88411-4f06-4724-a7a1-d7078a617626 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.786932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 249/503] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:33.786932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d88411-4f06-4724-a7a1-d7078a617626 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.807081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.807697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.807980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.808251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.829518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.881281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.882762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.893109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d88411-4f06-4724-a7a1-d7078a617626 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.903575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.926343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.926940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.927218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.927491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.928122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.928424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.929807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.941654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 250/504] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.942432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.980287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 255/505] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:33.981421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.032939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.046077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28eac1-1ae9-4b9c-8254-a7ee98801161 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.082778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d1341-d1a2-4cf6-8737-76ef7c7f1b32 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.085891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.085891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.085891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.085891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.096240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.097737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-e29b0ce5-f391-43ea-b890-f360f1467456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:34.112243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.113226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:34.114092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:34.114092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.114092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:34.114092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 251/506] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.114849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.114849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.115080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.119880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.120480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.120757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.121068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.121707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.122007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.123359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.133034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:34.134233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:34.135151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 256/507] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.136944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.151848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.203398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 252/508] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:34.204231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.237950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684446-05ce-4ac8-90a8-0c74c7ae4bea None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.248076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.273933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.274563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.274845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.275121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.282734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.284193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:34.296835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.297785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:34.298689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 257/509] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.299249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.316297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3293c-7057-4b01-95fd-239d76e91216 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.352420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.353483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.353775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.354047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.365188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.366698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:34.381886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.382797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:34.383654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:34.383654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.383654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:34.383654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 253/510] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.383880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.391917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.408205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347171a8-ccf9-41e0-a7ad-d6b918d6b928 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.447148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.447751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.448034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.448317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.448944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.449498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.450686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.467142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 258/511] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.467142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.497742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.498598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085e855-92b9-4548-b1ba-a6a967f79eeb None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.531966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.532581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.532858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.533174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.540340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.541779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:34.555074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.556227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 254/512] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.557196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.574789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f0a0-6256-4d45-9f06-b7e3afbda6a3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.611128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.611823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.612110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.612383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.613132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.613439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.614929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.620292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.632478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 259/513] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.633415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.661599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb148e5-4860-49b7-8350-edef102edbde None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.694354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.694937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.695237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.695509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.696399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.696699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.698108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.709707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 255/514] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.710672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.722309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.736092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4791d1-e247-42fa-96b8-922576472c43 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.770036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.770660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.770972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.771231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.778075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.779520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:34.791404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.792334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:34.793288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 260/515] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.793863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.813471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a950c8-433c-42e5-bf15-7f5736bfd317 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.836434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.848170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.848793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.849116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.849390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.860249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.861791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:34.878292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.879484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:34.880776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:34.880776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:34.880776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:34.880776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 256/516] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.880776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.898129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f923c30-66f4-42d9-ac09-b55b6d9a3e27 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.931143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.931743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.932026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.932297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.933039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.933347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.934772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.952528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 261/517] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.953347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.961567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.985291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd07f1f-0835-417a-8e82-133e6637b46a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.017858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.018460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.018743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.019017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.049870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2cc56-9d8a-4cb4-a5b6-64f38fff7814 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.063604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.084602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.085361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.085708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.086050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.090848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.092408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.093399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.094831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:35.106615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:35.107464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:35.108316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 262/518] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:35.109367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.172855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.195583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 257/519] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:35.196685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.209797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e60d7-bfd2-4e9a-bbf8-05ce4648c16d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.243585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.244122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.244404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.244680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.245427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.245731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.247168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.264654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 263/520] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.265509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.273760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.298592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d8f2d-5077-4e7d-a2f5-4b76eac9fbc1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.333170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.333771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.334052 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.334326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.334960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.335268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.336633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.348031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 258/521] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.348805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.366605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68bd2-09d8-42f7-a835-44744e25b325 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.390366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.401486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.402094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.402374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.402649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.448169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2249e3-b364-4c0a-aa06-bb31a15b5caa None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.471102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.472563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.480729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.481440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.481787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.482128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.482881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.483253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.485370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.489318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.504074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 259/522] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.504804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.579521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 264/523] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:35.580753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.605343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f412db-77aa-4a7e-8126-e95cdf95c193 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.606249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.638953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.639572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.640352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.640621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.650400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.652431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:35.667352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:35.668480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:35.669345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:35.669345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:35.669345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:35.669345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 260/524] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:35.669345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.681807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a520040-8e1b-4151-bdab-802b315a2409 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.715815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.716503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.716790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.716902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.717390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.767323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc0cb-4976-4b46-9aa9-af16052fa0c3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.785245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.786723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.801004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.801637 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.801922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.802198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.802834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.804242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.809771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.825316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 261/525] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:35 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:35.826044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.911823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 265/526] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:35.912926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.943518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676961c-ff38-4f38-9e6b-0b3118725883 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.947059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.981773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.982451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.982735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.983007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.983762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.984070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.985532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.991398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 262/527] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.992339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.052203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288d7b9-24a9-4a79-918b-21eea51c005b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.088436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.102346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.102996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.103276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.104204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.119276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.121165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.133808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c0ee-e88e-400c-b7b5-599debcab5eb None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:36.147590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:36.148564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:36.149477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:36.149477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:36.149477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:36.149477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 266/528] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:36.150554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.187398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.188439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.188864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.189541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.190886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.191399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.193915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.209127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 263/529] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.209979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.240535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.270809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bbcbb-7738-441f-8dff-3c539e2f573a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.310724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.312279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.312786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.313572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.325095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928488-86b0-4ccd-a94c-de8fac645544 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.329858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.331592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:36.351246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:36.352234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:36.353100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:36.353100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:36.353100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:36.359624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 267/530] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:36.359624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.378614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.379231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.379531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.379828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.380430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.380735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.382194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.395624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.395795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 264/531] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.396721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.493591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.498377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789396-0ac2-41fb-a3ad-7fad20460fd8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.508079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c703a-06e3-419a-a403-e0ffa23dc081 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.548106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.549096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.549558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.550013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.551290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.552020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.552385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.552655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.553466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.554073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.555301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.559424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.561146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.573046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 265/532] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.573046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:36.582228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:36.583097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.583975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:36.587200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 268/533] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:36.588157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.689102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed82d-214a-4739-977f-1d9cd375a96c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.709877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.724331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc85c2-f3a5-487c-8852-3670bea327a7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.738102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.739596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.740534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.741018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.751240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.753490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.762116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.762868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.763223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.763688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:36.769131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:36.769970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.770823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:36.773021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 266/534] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:36.773148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.774059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.775118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.787197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:36.788310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:36.789762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:36.790816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:36.797015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 269/535] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:36.797015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.903499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e51844-cd02-4590-a2b8-9895d7279298 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.920449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736d320-a42a-4110-ac5e-aa09ed83d7f5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.940522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.941208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.941527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.941749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.942495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.942804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.944322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.964087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 270/536] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.965091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.965835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.966695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.967209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.968685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.970786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.971400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.973129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.978376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.993042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 267/537] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.994178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.063380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.086025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e410ab-2491-44f0-ae6a-d0297a4b3663 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.107622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75646e4-ff80-4748-bb02-92fa066fb864 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.125069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.125686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.125971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.126251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.127110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.127481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.129159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.148003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 271/538] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.148003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.150679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.151314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.151591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.151862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.152608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=1085dcce677340bca92684d362e2fe7c) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.152917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.154459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.164321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 268/539] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/role_assignments?user.id=1085dcce677340bca92684d362e2fe7c&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.165193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.248170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.287917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a84c1-f48e-47e7-b367-b4dcd0765d64 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.302474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764006-897c-4251-9c71-67efbbf2c08c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.343999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.347423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.348695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.349788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.351284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.356754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=6b91846cfa8f40109e3d3d10dec3adc5) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.356754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.356754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217c67-acd9-412f-a4cb-c3a345d53b66 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.358231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.359516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.360347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.360852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.361861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=87ebde2ec14c40bab8a795230a1c5b42) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.362387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.364063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797df2-9224-40a3-98be-e9475375bdde None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.369193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 272/540] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/role_assignments?user.id=6b91846cfa8f40109e3d3d10dec3adc5&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.378390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 269/541] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/role_assignments?user.id=87ebde2ec14c40bab8a795230a1c5b42&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.833536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177470e0-15d0-4460-80c6-ef930de78358 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.833536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177470e0-15d0-4460-80c6-ef930de78358 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.833536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177470e0-15d0-4460-80c6-ef930de78358 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.833536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 273/542] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:37.840174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4612b-7fb1-4d44-bd6c-a30c243b11e1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.841534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4612b-7fb1-4d44-bd6c-a30c243b11e1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.842295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4612b-7fb1-4d44-bd6c-a30c243b11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.866732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-86b4612b-7fb1-4d44-bd6c-a30c243b11e1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:37.868268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-86b4612b-7fb1-4d44-bd6c-a30c243b11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:37.954266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 270/543] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:38.305536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bca0e3-8813-49ff-8677-18a7ee0ee6a5 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.305895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bca0e3-8813-49ff-8677-18a7ee0ee6a5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.306097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bca0e3-8813-49ff-8677-18a7ee0ee6a5 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.307664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 274/544] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:38.312950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.313255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.313529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.336651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:38.337899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:38.404904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bca0e3-8813-49ff-8677-18a7ee0ee6a5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.438090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 271/545] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:38.531227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bca0e3-8813-49ff-8677-18a7ee0ee6a5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.571249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.580837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.581800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.582282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.582758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.597190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:38.602357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:38.603232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:38.604029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:38.606637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 275/546] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:38.612662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.726652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.795632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c03c7-483d-482e-b6a3-7d667cbb74a3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.808412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.839338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.840267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.841208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.841707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.843593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:38.844259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.845894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.855974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 272/547] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:38.863198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.979430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0737d-f08b-44c3-aba0-a312a30b19d5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.047109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.048345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.049622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.050677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.053296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02528c7-a010-4f1b-8fdc-499d6456c428 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.066446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:39.073776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:39.074689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:39.075479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:39.078727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 276/548] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:39.084433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.091540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.126365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.126934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.127212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.127492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.153834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.155331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:39.187505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:39.189025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:39.190061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:39.190061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:39.190061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:39.190061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 273/549] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:39.200039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.201746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.290830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d37a-e43d-455d-aaa8-0a54594a34c4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.330437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.331254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.331678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.332046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.332815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:39.333246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.334722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.344332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 277/550] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.350025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.379700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e303399-0bd9-405b-b698-e4f73f065802 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.447513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.448132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.448410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.448683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.449356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:39.449663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.451025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.470115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 274/551] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.476146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.545323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.554775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0abc9-4c45-46f9-9a9e-2353c015224b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.581323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.612140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.612796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.613108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.613399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.634767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.636179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.640254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a6bc2-a25a-430f-9db1-08bd1df78767 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-85c3e8ad-60a3-432a-b80b-afcbef882604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:39.673068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:39.674219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:39.675081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:39.675081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:39.675081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:39.675081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 278/552] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:39.680045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786aaa1-2cdd-4c94-a1e8-a2032cf26419 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.680176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.680274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786aaa1-2cdd-4c94-a1e8-a2032cf26419 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.680274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786aaa1-2cdd-4c94-a1e8-a2032cf26419 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.680437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.680437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.680437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] PATH_INFO: `/v3/users/swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.680617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 279/553] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.681009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786aaa1-2cdd-4c94-a1e8-a2032cf26419 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.686250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.688449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-43a4886f-17b9-40f7-b0a2-f949ee4ffaec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:39.701107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:39.702092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:39.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 275/554] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:39.705211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d67050-9c71-4135-9c1f-f29301ddd980 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.705211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d67050-9c71-4135-9c1f-f29301ddd980 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.705211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d67050-9c71-4135-9c1f-f29301ddd980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.723804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-25d67050-9c71-4135-9c1f-f29301ddd980 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:39.725066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-25d67050-9c71-4135-9c1f-f29301ddd980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:39.814505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786aaa1-2cdd-4c94-a1e8-a2032cf26419 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.815056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 276/555] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:39.817124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881d40-9203-4cf6-bc57-3fa5b1ad6f84 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.817596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881d40-9203-4cf6-bc57-3fa5b1ad6f84 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.818058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881d40-9203-4cf6-bc57-3fa5b1ad6f84 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.819950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 277/556] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.821402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881d40-9203-4cf6-bc57-3fa5b1ad6f84 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.850695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.857759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.859028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.859314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.859618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.860260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:39.860564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.862234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b571da-483f-4428-8bd1-579b13266a15 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 280/557] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb44-a13a-4179-9453-880a69b0c814 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb44-a13a-4179-9453-880a69b0c814 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb44-a13a-4179-9453-880a69b0c814 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 281/558] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.875157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.880151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.880151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.898807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:39.899936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:39.922851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.941255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881d40-9203-4cf6-bc57-3fa5b1ad6f84 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.976599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 282/559] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:39.976599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.987625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.988416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.989099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.989381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.990173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:39.990482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.992059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af26e0-77dc-456c-baf7-d6a8e9a35207 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.030987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 278/560] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.032595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.033057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.033481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.051347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:40.052830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:40.112414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde66af-a811-43d6-9b33-995809f2cf9f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.136262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 279/561] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:40.137168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.156208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.157122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.157576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.158035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.160160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.174289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.176545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:40.192132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.193164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 283/562] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.194141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.216772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.276035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6142-c2bf-43d3-9668-e0b14fab8344 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.312378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.313010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.313297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.313570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.326235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caab5db-4f89-4f9f-b9bf-821b0c91a410 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.365197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.365871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.366148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.366467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.381104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:40.386660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.387801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 284/563] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.388837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.393902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.395679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.406806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.470664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.494587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46d7f-ced7-43e0-a24f-df429a692d3f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.500830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 280/564] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:40.501694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.532527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.533157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.533441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.533713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.534428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:40.534728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.536170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.554888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 285/565] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.555685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.605258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a842e-6e47-4560-ae1a-5a0f10f0bae7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.614991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.640602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.641256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.641538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.641809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.651652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:40.654455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5a68b-3f79-4ee4-bff8-e879197f2a22 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:40.657028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.657926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.658798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:40.661100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 281/566] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.662093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.691078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.691771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.692048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.692379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.693148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:40.693543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.695052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.696696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.704395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 286/567] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.705232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.766328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed553865-7b30-44ce-a20b-41f74f2331ee None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.801082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.801712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.802015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.802345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.812263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13541d3-fd83-4ab0-a040-4ee75f491d71 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.812570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:40.817403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.818362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:40.819210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 282/568] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.820162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.851847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.856352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.857012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.857295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.857567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.870324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.871908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:40.886843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.887848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:40.888750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:40.888750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:40.888750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:40.888750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 287/569] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.888750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.931831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011c73b-d6c1-46a1-a9d3-77d5e24713b2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.938770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.967296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.967952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.968241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.968523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.978737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.980181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.989549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf77dff-628a-4c26-b067-6054f84ed371 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:40.995228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.996180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:41.002404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:41.002404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.002404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:41.002404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 283/570] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.002404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.023830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.024429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.024710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.025013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.025666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.025971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.027344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.035437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 288/571] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.036136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.081321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.101085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1e3c-899f-4b90-b20e-d2501f399961 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.136242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.136868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.137180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.137459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.139551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bf1a3-8f04-4422-8cab-529ced1ef65d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.148347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.149851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.150049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:41.164164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.165362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:41.166437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:41.166437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.166437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:41.166437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 284/572] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.166437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.176775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.177428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.177708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.177983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.178644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.178946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.180386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.189351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 289/573] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.190083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.272510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c4614-b482-4e29-90b7-4523a8a5c875 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.302037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d989e-d252-4e23-8392-55e162762294 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.302568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.307740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.308345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.308624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.309242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.309564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.309868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.311229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.323292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 285/574] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.328548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.335757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.336370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.336660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.336935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.337612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.337915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.339296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.356315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 290/575] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.356315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.382705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.432008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbab34-5324-4438-ae1a-7fb371486d0d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.450978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e708556-bced-4456-85fd-bbd6c25be1f8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.467244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.468441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.468718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.469050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.478975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.480607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.485333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.485982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.486313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.486635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.487370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.487672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.489320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:41.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.496899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:41.497833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:41.497833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.497833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:41.497833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 286/576] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.497833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.509216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 291/577] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.509216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.544131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.599078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ff4af-a381-4bf9-b22b-068437fbe9c7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.600121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.610160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665fc3-f67c-43cc-bdca-9b812b59c165 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.633067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.633715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.634039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.634359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.643194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.643823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.644101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.644372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.645580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.647079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.651033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.652471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:41.661651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.662617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:41.663528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:41.663528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:41.663528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:41.664048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 287/578] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:41.664922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.665815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:41.666726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:41.667163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.667235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 292/579] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.667235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.764347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.767236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161973a2-cd4e-460b-9abf-fa9bfc397e5f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.771487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc395-8eb1-44b5-9adf-441b8904ccd9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.800120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.800718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.801028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.801302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.801936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.802231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.803606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.805292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.805921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.806188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.806456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.813455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.814983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.817735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 293/580] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.818620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:41.827675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.828763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 288/581] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.834438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.835695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.918489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb543-6a83-4aed-bef9-0523c8340119 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.932025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb928-238d-42df-a65d-9f3b478a8e57 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.951695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.952847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.953155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.953428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.960294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.961759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.965375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.965982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.966269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.966542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.967180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.967482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.969477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:41.973586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.974664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:41.975812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:41.977096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 294/582] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.978082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.981471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 289/583] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.982409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.001420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.047967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.078690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b294-000c-43b8-887f-a425874ae0db None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.084488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ed490-69e5-4139-8b3c-bcf0ffbbb611 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.115063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.115773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.116059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.116462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.117165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.117475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.118923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.121374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.122005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.122288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.122563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.130413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.131708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 295/584] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.131855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.132505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:42.144280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:42.145492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 290/585] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:42.146519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.224737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.236749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb19ec-4747-494e-9df4-c3339b14c553 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.247923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bb3c1-e65a-45ee-b380-e732f8824f1e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.267340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.273358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d50de-3368-439c-9a8c-10312125006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.273966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d50de-3368-439c-9a8c-10312125006c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.274247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d50de-3368-439c-9a8c-10312125006c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.274519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d50de-3368-439c-9a8c-10312125006c None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.275202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d50de-3368-439c-9a8c-10312125006c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.275505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d50de-3368-439c-9a8c-10312125006c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.276971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d50de-3368-439c-9a8c-10312125006c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.282585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.283234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.283514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.283788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.284536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.284838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.286367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.288431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 296/586] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.289250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d50de-3368-439c-9a8c-10312125006c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.313266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 291/587] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.313266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.391155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d50de-3368-439c-9a8c-10312125006c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.409042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7239c62-a395-4831-b59e-a07164c36423 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.425001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.425625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.425904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.426175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.433388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.434815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.443301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.443901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.444187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.444464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.445214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=24659ce97c204823a8968c757ea40e56) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.445524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:42.447028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:42.448095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:42.449353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:42.450005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.450376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.453136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 297/588] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:42.453687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 292/589] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/role_assignments?user.id=24659ce97c204823a8968c757ea40e56&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.453981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.454965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.516463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.576020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a063f-f7f1-45e8-a6d7-58dba17d7946 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.580522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381c2d-ab8d-4e20-b61e-324842f6d5e8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.616006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.616724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.617103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.617445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.618238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.618630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.620072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.628236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.629000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.629570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.629920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.643184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 298/590] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.643184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.653324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=ad5260213b9f4ec49f3ce2ec48945d7d, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.655631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.672993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 293/591] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:42 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:42.674045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.749998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.762519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247d42-5013-45ed-a5d1-7e670ca05739 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.776390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.806945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.807993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.808409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.808827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.810450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b28420-d953-4de5-b464-7efd07a2d513 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.820604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.822761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:42.839631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:42.840724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:42.841805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:42.841805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:42.841805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:42.843917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 299/592] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:42.844873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.853664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.854336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.854622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.854905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.856089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.856089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.857437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.883757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 294/593] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.883757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.970392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd5367-ef26-400f-b460-091d54921650 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.006663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.007481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.007684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.007961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.019211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f7e0-1f66-4ba3-863c-773c886a0f3c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.019719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.021222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.029070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:43.037974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.039151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:43.040199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:43.040199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.040199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:43.043802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 300/594] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:43.044648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.063388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.068871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.069846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.071577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.072006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.073182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.073628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.076026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.091653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 295/595] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.092332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.181271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e770f3-d8f5-4fc6-926b-327e65148e49 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.210361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ce14f-41e7-4f72-914c-f6c174493897 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.219675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.220596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.220876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.221189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.233605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.235187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:43.250860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.252282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:43.253218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:43.253218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:43.253218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:43.256035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.256035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.256035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.256807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.256989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 301/596] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:43.257154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.257302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.260978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.261196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.275610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 296/597] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.276426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.307514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.351679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.390583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa180b0-e842-4396-aa64-fb08d716aa70 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.395980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f2f7e-f0e4-4eaf-a125-29e366dfb4e0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.429368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.430825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.431346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.431762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.438039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.438652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.438932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.439205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.439844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.440149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.441499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.441620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.443458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.454277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 297/598] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.455087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:43.458450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.459498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.460457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:43.462656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 302/599] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:43.464519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.551344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.569803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71c83-b06d-4f7c-88a3-b01ec1e44ef6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.585628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e59a-4f23-4a85-93bb-b41db711e919 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.610938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.611589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.611982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.612424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.615499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.620061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.622028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.625931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.626730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.627158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.627603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.634777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:43.635399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.636320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:43.639143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.641592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 298/600] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:43.642763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:43.652599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:43.653548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:43.654440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:43.661862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 303/601] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:43.663177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.751647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19af3f0-26a3-456b-b346-99e7a44a4129 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.770321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4841049-044e-46c4-bb7d-630274156d60 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.787890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.788603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.788886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.789189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.789879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.790289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.791761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.808992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 299/602] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.808992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.814020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.815082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.815862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.816415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.816994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.817955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.818480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.820178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.836241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 304/603] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.837541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.897841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.929896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f5371-433e-4d9c-bbef-1b1da1a31423 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.970094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.970295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a3b52-e335-4614-8557-41358883e174 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.971149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.971703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.972155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.973160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.973641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.975315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.988998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 300/604] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.990306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.010799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.011844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.012128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.012406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/87ebde2ec14c40bab8a795230a1c5b42/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.037865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=87ebde2ec14c40bab8a795230a1c5b42, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.039893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.057642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 305/605] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:43 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/87ebde2ec14c40bab8a795230a1c5b42/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:44.058619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.071856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.122838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b229af-d4e8-4e52-a59e-3481e9079f21 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.167071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.167668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.168071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.168289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/6b91846cfa8f40109e3d3d10dec3adc5/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.185723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=6b91846cfa8f40109e3d3d10dec3adc5, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.187206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1bc24-33f5-4245-bd3c-f17709c65ca8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.211593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.218949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db29ef-cbae-4e22-aaf3-d786b467e424 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.235737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 301/606] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:43 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/6b91846cfa8f40109e3d3d10dec3adc5/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:44.265084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.265837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.266423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.266701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/1085dcce677340bca92684d362e2fe7c/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.284182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=1085dcce677340bca92684d362e2fe7c, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.285701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2bfc-6478-4a9f-957f-1a54f995ddeb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.305085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 306/607] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:44 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/1085dcce677340bca92684d362e2fe7c/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:44.450605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed28ff-fdad-4c23-b5de-4124ac685e8b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.451125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed28ff-fdad-4c23-b5de-4124ac685e8b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.451396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed28ff-fdad-4c23-b5de-4124ac685e8b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.453754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 302/608] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:44.459125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ffc0a-9584-4fd8-b4a7-5f3959440818 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.459414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ffc0a-9584-4fd8-b4a7-5f3959440818 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.459691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ffc0a-9584-4fd8-b4a7-5f3959440818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.482962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-931ffc0a-9584-4fd8-b4a7-5f3959440818 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:44.484245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-931ffc0a-9584-4fd8-b4a7-5f3959440818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:44.580988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 307/609] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:44.860746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1917b5-0b9c-4331-af53-8881793e4ad2 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.861012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1917b5-0b9c-4331-af53-8881793e4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.861335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1917b5-0b9c-4331-af53-8881793e4ad2 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.863405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 303/610] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:44.868769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.869096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.869379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.898002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:44.899223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:44.995429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 308/611] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:45.039648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1917b5-0b9c-4331-af53-8881793e4ad2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.202783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1917b5-0b9c-4331-af53-8881793e4ad2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:45.223598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:45.252949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.253608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.253890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.254223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.273161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:45.278182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:45.279156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:45.281125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 304/612] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:45.285849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.299302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:45.438673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadb5d7-6a31-42d4-a01f-e1358e484bb2 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:45.498665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.499678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.500160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.500651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.501546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:45.502053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:45.503661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:45.508224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.517005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 309/613] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:45.528597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.654393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:45.718137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e59d9-9adc-4c92-9a44-21f4b8d33255 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:45.722385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7dddf-0956-4ff2-8f30-1f5e28abc76d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:45.788438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.789091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.789431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.789712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.790808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.791432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.791933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.792211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.812412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:45.813900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:45.814894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:45.821813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:45.822738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:45.823589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:45.824740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 305/614] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:45.830879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:45.847900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:45.850312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:45.852397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:45.852397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:45.852397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:45.852397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 310/615] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:45.857641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:45.909607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:45.978860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b948e-1019-4fa3-8f3f-1d93de8346b3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.015267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.015996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.016285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.016560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.017358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:46.017667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.019023 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.028204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 306/616] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:46.034144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.074077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31d92-69f5-4b7a-a02b-4565d7f5de4f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.129200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.156219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ab0ff-371d-438d-94be-1e4f8b5b96a8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.161358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.178623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.179394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.179777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.180220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.180984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:46.181390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.182860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.203495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.204741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.205247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.205689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.210855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 311/617] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:46.221845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.223260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.233046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:46.258275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:46.259375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:46.260494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:46.260494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:46.260494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:46.268070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 307/618] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:46.273947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.395663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa352d-acc6-42a5-aa41-0f350f86aa2a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.435837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c8c3c-0832-4500-a1f8-4306a9cc66ce None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.439337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.439972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.440248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.440517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.448625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.450098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-daf58fde-cce6-4927-b485-f8961b1987b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:46.463364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:46.464524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:46.465709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 312/619] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:46.466329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd335-97f1-45e3-bc03-1276a4e8a848 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.466329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd335-97f1-45e3-bc03-1276a4e8a848 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.466657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbd335-97f1-45e3-bc03-1276a4e8a848 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.468475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 313/620] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:46.469633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d78e4-d959-458f-bda8-53e1ff981cee None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.469738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d78e4-d959-458f-bda8-53e1ff981cee None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.469937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d78e4-d959-458f-bda8-53e1ff981cee None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.471392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 314/621] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:46.472175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d78e4-d959-458f-bda8-53e1ff981cee None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.487706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.488550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.488832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.489157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.489803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:46.490107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.491472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9574f6-d1d0-4fb4-99d4-a523baacd260 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.508696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 308/622] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:46.510015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905cf5c-ee4b-4d84-8cef-589dff1731cc None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.510307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905cf5c-ee4b-4d84-8cef-589dff1731cc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.510584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905cf5c-ee4b-4d84-8cef-589dff1731cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.522480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.529596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f905cf5c-ee4b-4d84-8cef-589dff1731cc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:46.530827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f905cf5c-ee4b-4d84-8cef-589dff1731cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:46.534655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.594977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d78e4-d959-458f-bda8-53e1ff981cee None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.608996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 309/623] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:46.608996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.608996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.608996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.631888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.632502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.632784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.633091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.633815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:46.634119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.635174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:46.635566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.636457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:46.660127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 315/624] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:46.661673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.726648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 310/625] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:46.727809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.817348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.824765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066695d3-f1b1-4188-a9af-314c23dcd364 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.855024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:46.863300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.863938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.864220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.864493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.871761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.873251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-092fbd70-2435-4adc-8ab7-bd6b11322f9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:46.885340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:46.886302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:46.887246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 316/626] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:46.888015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155a7fc-b12c-4c74-9022-55b88c961626 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.888320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155a7fc-b12c-4c74-9022-55b88c961626 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.888597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155a7fc-b12c-4c74-9022-55b88c961626 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.893040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 317/627] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:46.893957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155a7fc-b12c-4c74-9022-55b88c961626 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:46.897148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518d436-1f75-4bc7-bcc7-65db29163f10 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:46.934082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:46.934732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:46.935004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:46.935273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:46.946353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:46.947801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:46.962556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:46.963522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:46.964452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:46.964452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:46.964452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:46.968412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 311/628] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:46.969292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.004572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155a7fc-b12c-4c74-9022-55b88c961626 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.039508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.040134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.040414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.040685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.051208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-040c3e70-e143-4718-86b4-cc68808914a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:47.056408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.057494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 318/629] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.062946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.063786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.063786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.074353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c64-692a-4179-b229-6155e6c9be29 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.078159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:47.079359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:47.092692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.098118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.110367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.110973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.111253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.111525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.112248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.112552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.114011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.131167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 312/630] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:33:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.131970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.146138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 319/631] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:47.146967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.245069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f178f0c-8f45-4fde-a3d9-db029dbeee0d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.253264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a451-2016-4306-8846-d89885a5f29b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.281801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.282500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.282783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.283065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.283685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.283991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.285388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.290756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.292076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.292367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.292644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.302441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 313/632] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.302441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.307728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:47.312651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.313599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.314445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:47.318610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 320/633] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.319433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.330731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.350595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.413926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cef927-95b6-4a15-beb0-09d56bffa82a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.420393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa754b-bed2-48b5-8626-ccce752454d2 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.447404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.448014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.448296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.448573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.449266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.449574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.450918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.455080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.455680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.455965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.456240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.463325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 314/634] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.463325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.522845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.524345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.534566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.561567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f8525-fe96-4f3a-bdff-946f936b006f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.563140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.597390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.598021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.598313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.598587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.605285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.606744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:47.618463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.619489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 315/635] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.620401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.627868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 321/636] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:47.629130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.720000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300818-1e50-4efb-a66d-efa65083e3fe None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.728828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94327653-2318-4f14-b86e-f84f39641540 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.737891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.752716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.753341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.753622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.753897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.754551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.754856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.756196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.762054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.762678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.762960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.763235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.763945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 316/637] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.765253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.778011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:47.783352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.784460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 322/638] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.785502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.791964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.865621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647952-d1f9-4136-9476-df008bb82142 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.888426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0d4e2-1396-4e8b-a602-db304e749af8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:47.897999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.898616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.898894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.899169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.909786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.911216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.922215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:47.922813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.923091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.923360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.924034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:47.924335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:47.924902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:47.926009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:47.926919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:47.926919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:47.926919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:47.927093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:47.927172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 317/639] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:47.927172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.941017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 323/640] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:47.941681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:47.970032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.009990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.033513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c83a2-7ae6-4f65-b41a-c18865e54633 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.040587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a46c-ebea-43b4-adbb-df9d8e25d055 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.066788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.067416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.067698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.067973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.078166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.078802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.079086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.079289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.079384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.080733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.090202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.091669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:48.095189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.096360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:48.097310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:48.097310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.097310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:48.097310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 318/641] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.097552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:48.105875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.107030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:48.108193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:48.108193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.108193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:48.108193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 324/642] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.108193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.190917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.204551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635f33-d976-46c9-a456-bb78adef7475 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.212925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c667000-bf97-48d4-a8ae-0473328b3477 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.230194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.238534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.239138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.239422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.239702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.240358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:48.240650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.242050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.245330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.245924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.246200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.246468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.247093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:48.247390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.249301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.250435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 319/643] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:48.251193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.266658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 325/644] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:48.266658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.351898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998c31e-29a8-4d10-b6a5-86c66df714a9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.361646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60a52-9748-4b3e-a305-6cb9b8ddf78a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.386653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.387309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.387593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.387869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/24659ce97c204823a8968c757ea40e56/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.395854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.396446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.396726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.397018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.397644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:48.397943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.399301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.402886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=24659ce97c204823a8968c757ea40e56, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.404296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.407625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.410871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 326/645] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:48.411653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.421287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 320/646] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:48 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/24659ce97c204823a8968c757ea40e56/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:48.422100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.470547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.530074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe1364-e551-44d3-b6c7-de2178d6d9fc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.537922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe69777-b21d-40a4-beac-84548b8bae3e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.570723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.571608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.571894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.572169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.572812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:48.573147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.574520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.580433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.581176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.581461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.581742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.586889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 327/647] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:48.587729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.593053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.594492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:48.612612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.613790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:48.614682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:48.614682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:48.614682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:48.614682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 321/648] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.617206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.656480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.707110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff653c29-3cef-4a9d-8dfb-0acad2929bb5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.718947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.730897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4073d2-edc5-44d6-8d6d-c969f6a7bb2d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.752386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.753057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.753343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.753620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.762637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.764905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.770155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.770791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.771073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.771349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.778154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.779589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:48.780790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.781923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:33:48.783074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:48.785650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 328/649] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.786572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:48.794157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.795214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 322/650] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.796715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.897419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2db2-3732-40d5-b820-b04636eefd0f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.903857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3720a4e-66c4-4e60-800e-3104123fadd2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.917467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.932440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.933114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.933410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.933673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.941662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.942123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.942733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.943016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.943113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.943293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.944117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:48.944426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:48.946319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:48.954498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.955464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 329/651] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.956485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.967343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 323/652] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:48.967343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.980227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.073722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b56a3-0889-41c6-a540-6f19daca4e77 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.075706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7092cc-4577-4f87-921b-454b7f4c0103 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.115011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.115623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.115916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.116180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.116873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.117218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.118484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.118631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.119366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.119793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.120223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.121152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.121631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.123191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.130456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 324/653] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.131074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.147250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 330/654] 10.222.0.25 () {56 vars in 1139 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.148341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.194854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.211343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.242832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40619db4-cfab-4b7a-8dd5-0ec9b02676e4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.259438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70d87a-8689-42c9-8d62-3db9faa0959b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.279276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.279971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.280725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.281033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.281762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.282072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.283502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.296135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.296767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.297076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.297350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.306516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 325/655] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.306516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.306743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.306743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:49.318060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:49.319056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 331/656] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:49.320038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.420523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e172e2-2e54-4aaa-8f06-2952763390cc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.426207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.431032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594b6d8-8736-4808-b77d-ef119e083f92 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.451584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.458620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.459237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.459519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.459798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.460515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.460823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.462291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.468744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 326/657] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.472996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.480271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557768e-ba07-4600-896a-495025a68411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.481022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557768e-ba07-4600-896a-495025a68411 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.482326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557768e-ba07-4600-896a-495025a68411 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.482709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557768e-ba07-4600-896a-495025a68411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.588055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557768e-ba07-4600-896a-495025a68411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.589720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557768e-ba07-4600-896a-495025a68411 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.601355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70050fd6-dd94-47eb-b7f4-d6678e6168ef None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.644486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.645452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.645932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.646401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.721262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.721714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.727942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 332/658] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:49.729867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557768e-ba07-4600-896a-495025a68411 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.735356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.736902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.841864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557768e-ba07-4600-896a-495025a68411 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.871556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 327/659] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:49.872821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.883111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.884038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.884600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.885143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.885955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.886335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.887884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.912301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 333/660] 10.222.0.25 () {56 vars in 1137 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.913200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.973989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.997348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.006638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b111e-8ba2-4538-a606-60f58a339330 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.021695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a13c62-9739-44d3-b836-11e026d0f89d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.055514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.057784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.058370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.058530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.064284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.065190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.065662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.066121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.072598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.074245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.079225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.080858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:50.091010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.092020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:50.093053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:50.093053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.093053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:50.097287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 328/661] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.098308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:50.104580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.105926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:50.106957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:50.106957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.106957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:50.109549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 334/662] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.111067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.214969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb53eb-eb00-4dc3-bf3a-2cf45ff0ce81 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.253849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.254531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.254915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.255294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.256536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4a5e-1a91-4e94-8dba-9fd5eeb58a16 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.267003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.277507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.301778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.303521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.303998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.304452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.305306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.305803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.307849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.324195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 335/663] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.325684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.345632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.348279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.439250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14801754-5002-49f1-8294-f030f7f0ad38 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.461636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 329/664] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:50.462399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.479885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.480784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.481115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.481397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.482037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.482344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.484359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf240c-c28d-4f86-975e-361729b66b79 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.496227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 336/665] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.497432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baae3-de14-45f9-b794-e192a5d0f9a5 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.497722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baae3-de14-45f9-b794-e192a5d0f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.498002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7baae3-de14-45f9-b794-e192a5d0f9a5 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.503373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 337/666] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:50.504190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7baae3-de14-45f9-b794-e192a5d0f9a5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.518860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.535655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.581142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c7e93-d770-4c64-ab80-ae443ca5371c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.614561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7baae3-de14-45f9-b794-e192a5d0f9a5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.617579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.618271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.618551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.618826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.627186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.628698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:50.641816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.642782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 330/667] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.648599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.658635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.659387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.659768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.660145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.671063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.672616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-031b7c90-6267-4df3-9aa2-b63b240ef7b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:50.687214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.688178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 338/668] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.693959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.709213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:50.710456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:50.762810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199e91e-d7da-4eb6-81d1-bbea945206b9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.770955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.781094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 339/669] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:50.782060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.794287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.814946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.815912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.816380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.816947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.824520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.826368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:50.840456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.841397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:50.842293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:50.845908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 331/670] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.847354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.888899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804434-eddc-45d7-af6e-a907e074d71c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.923683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.924370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.924721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.925086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.925846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.926230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.927720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.939808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 340/671] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.940832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.962229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033b84-90f1-42e6-94e9-d2a78f2345a1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.999311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.999985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.000381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.000664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.001341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.001652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.003164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.013638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.022545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 332/672] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.023690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.045989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79871531-c3aa-43ba-a834-bfe658bb75c5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.054627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.083948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.084599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.084825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.085141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.085831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.086130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.087622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.105642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 341/673] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.106540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.141412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77ef98-e419-4130-b05a-26a83bff6a33 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.180313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.180916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.181855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.181855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.182211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=87ebde2ec14c40bab8a795230a1c5b42) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.182520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.183937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.200989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 333/674] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/role_assignments?user.id=87ebde2ec14c40bab8a795230a1c5b42&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.200989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.238333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200d1b0-462a-4c72-a650-04aafbed3136 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.273622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.280199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.281383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.282001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.282657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.290934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.292628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:51.306997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:51.308270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:51.309552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:51.311701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 342/675] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:51.312756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.327518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.339136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558556b1-6611-47e0-acbd-944e065106c6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.381739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.383394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.383773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.384185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.398637 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:51.404950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:51.405968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 334/676] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:51.406985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.438977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31946-1703-491d-a4bc-30b24f315504 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.477567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.478339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.478708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.479270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.479876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=1085dcce677340bca92684d362e2fe7c) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.480278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.481974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.491892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 343/677] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/role_assignments?user.id=1085dcce677340bca92684d362e2fe7c&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.492850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.548227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba410d-b49a-4a29-b3aa-54b84d5ef446 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.594051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.594618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.595412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.596145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.596448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.597422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.597908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.599580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec826-55ad-4112-8041-e44581d7443d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.617296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 335/678] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.623463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d795b-ea10-4e95-a435-59e8b8d159ad None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.623463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d795b-ea10-4e95-a435-59e8b8d159ad None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.623463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d795b-ea10-4e95-a435-59e8b8d159ad None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.625055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 336/679] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:51.628371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d795b-ea10-4e95-a435-59e8b8d159ad None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.647056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.659411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fd0ba-5b8c-4bdc-a00a-97ffdfefe841 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.711805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.712469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.712751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.713178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.713833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.714178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.715562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45f66-b04d-407c-b41f-489a93e538b7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.726998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 344/680] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.728264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.728552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.728832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.751534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:51.752720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:51.766470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d795b-ea10-4e95-a435-59e8b8d159ad None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.804291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.804986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.805511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.805878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.806708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=6b91846cfa8f40109e3d3d10dec3adc5) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.807115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.808655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.815840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 337/681] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/role_assignments?user.id=6b91846cfa8f40109e3d3d10dec3adc5&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.816747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.863690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 345/682] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:51.937170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.976898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1417467-0fdf-4f65-8978-b795f668d43b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.015729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.016364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.016647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.016923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.029139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.030619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:52.045091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:52.046107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:52.047112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:52.047112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:52.047112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:52.047112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 338/683] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:52.052063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.128188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.159685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9a90-4d34-4bce-87bf-0624dfba1ec5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.197359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.197986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.198269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.198545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.199255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.199566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.200946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.213398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 346/684] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.220079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.231399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.361318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826ea8-bbbf-4d29-a095-18d9d373cb97 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.362639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f27203-df6c-430a-a4f6-ab2a23cfbd0f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.370047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.391554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.400104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.400754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.401178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.401463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.403275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.404084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.404920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.405244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.409472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.410909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.417704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:52.423687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:52.424608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:52.425503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:52.426655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:52.428008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:52.432145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 339/685] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:52.432916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 347/686] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:52.435742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.435883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.561604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6d5f3-13cb-4968-8c9b-d15fb7375d0d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.566043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227579-27da-4091-9dfb-e2d65c9bd463 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.605268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.606207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.606695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.607143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.608062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.608713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.609129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.609294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.609582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.609857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.610587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.610895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.610996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.612379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.622695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 340/687] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.629374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.631148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 348/688] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.636526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.678473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.733382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.810549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250dc48-c301-47ac-b527-4e5c29ee7df1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.815872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7f7fd-0aaf-413b-aafc-1169604ef89f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.860194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.861043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.865046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.865046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.874779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.876499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.886578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.887414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.887836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.888279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:52.892594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:52.893750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:52.894845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:52.894845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:52.894845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:52.896418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 341/689] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:52.904252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.972246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.974029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.002827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.084320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 349/690] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:53.093186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.125473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.129536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fbcf0-238d-4e51-b0c2-f571d1044268 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.169319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.169319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.169319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.169319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.169756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.169756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.169756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.189789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 342/691] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.194807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.281106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a94f49-aca3-4633-8cc5-bdbbc5f0a3df None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.332682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e04d4-fca7-416c-926b-6861bebe236c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.337287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.379620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.379620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.379620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.379620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.393971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.395644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.405563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.406336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.406618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.406893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-f61a4bad-971f-486f-b02a-00a43d46ab7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:53.414031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:53.414989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:53.415889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:53.415889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:53.415889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:53.418050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.418277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 350/692] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:53.419503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bf98-6e55-4926-860b-026dde107a66 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.419787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bf98-6e55-4926-860b-026dde107a66 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.420063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bf98-6e55-4926-860b-026dde107a66 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.421302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.422064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 351/693] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:53.423775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4bf98-6e55-4926-860b-026dde107a66 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-bc1f70dc-4bf2-48db-a6e7-205d62c1a82f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:53.438266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:53.439283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:33:53.440234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:53.443240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 343/694] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:53.444285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.444572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.444850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.468543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:53.469839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:53.486193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.553111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 344/695] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:53.553998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.566162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4bf98-6e55-4926-860b-026dde107a66 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.627632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.628251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.628528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.628806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.629467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.629819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.631150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.655756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 352/696] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:53.661509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.708785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.727665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e43b2-878d-481c-84e9-735e6a9851a7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.764067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.764725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.765040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.765322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.766056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.766367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.767796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.786105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 345/697] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.791505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.815964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.871689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ee98b-fb53-4ab6-addb-140745c11964 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.894982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9091f7a-781b-4c27-93b3-018e0b60a833 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.923227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.923913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.924196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.924521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.930106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.930713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.930996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.931274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.932175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.933923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-b9eaf92e-6847-4fcf-bfa1-e4924b44522b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:53.946991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:53.948104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:53.949412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 353/698] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:53.950041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710e48-d5ce-47b6-84f7-98dab2775854 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.950041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710e48-d5ce-47b6-84f7-98dab2775854 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.950207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710e48-d5ce-47b6-84f7-98dab2775854 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.951349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 354/699] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:53.952080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710e48-d5ce-47b6-84f7-98dab2775854 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.010136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.011819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc077a-6b1f-47bc-949e-3d50a20870d3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.022319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.052693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.060431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710e48-d5ce-47b6-84f7-98dab2775854 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.106278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.106912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.107191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.107463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] PATH_INFO: `/v3/services/network` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.108710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.110231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:54.121155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:54.122210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.123066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 355/700] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:54.123903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.135217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 346/701] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:54.140722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.141045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.141328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.160668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:54.161911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:54.231765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc803-861e-474a-ba75-ac6ab1a6f74a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.237026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 347/702] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:54.238519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.275080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.275995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.276482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.277061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.278028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.278535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.280721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.288352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.297951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 356/703] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.299298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.345395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.368192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8ab8b-82da-4c9a-b83c-e071e01fcc8e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.416249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.417772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.418238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.418691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.419502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] RBAC: Adding query filter params (name=network) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.420081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.421815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.426669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905bf94-2544-49b8-a6ac-c64f94d5fd55 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.431850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 348/704] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.432918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.464912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.465427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.465707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.465981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.477203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.478794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:54.494112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:54.495102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:54.496004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:54.496004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:54.496004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.499917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 357/705] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:54.500826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.548182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49f1ea-ab57-4369-8467-77647b7097f1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.550172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.573761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.588790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.589450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.589741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.590017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.590751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=24659ce97c204823a8968c757ea40e56) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.591061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.593886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.599194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 349/706] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/role_assignments?user.id=24659ce97c204823a8968c757ea40e56&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.600126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.615725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843877e-5173-4c8e-9f3d-d888309ddd18 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.653254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.655462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.655462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.655462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.655805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] RBAC: Adding query filter params (type=network) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.656102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.657542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.671958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 358/707] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.671958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.740729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb686a-1073-4c12-8934-4d409d5f9244 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.790130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.790758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.791041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.791318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.792276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.792585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.794833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cc650-b09a-4852-a58d-9a6679027460 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.807544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3200cf-4f44-479a-b45e-42efe16faf46 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.813017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 350/708] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.813017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bee802-3204-445e-95eb-3aa8686d1398 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.813017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bee802-3204-445e-95eb-3aa8686d1398 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.813017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bee802-3204-445e-95eb-3aa8686d1398 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.813430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 351/709] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:54.819051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.819403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.819786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.839278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:54.840470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:54.850482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.851128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.851411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.851692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.852274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.854501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.859949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:54.866784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:54.868284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:54.869341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 359/710] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:54.870240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.870458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.935401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 352/711] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:54.937310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.038932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff179f8-e41a-47f1-8492-d1f1f93a49ab None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.059221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5452de-8a5a-4a24-9b6c-d4c8989de17b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.100406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.101058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.101434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.101717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.102462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:55.102778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.104166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.109239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 353/712] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:55.112485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.113228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.113577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.113920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.115962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.134006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.135478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.139987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:33:55.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:55.163399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:33:55.164396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:55.172408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 360/713] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:55.178506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.231901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcb768-de5c-44a5-a627-65bb5bcfbf0a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.260799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.271902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.272638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.272928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.273233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.273871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:55.274175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.275530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.286515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 354/714] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:55.286515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.350249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb67084-1f3d-4cd0-998f-deca7d8820e7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.404804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.405994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.406507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.406969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.408986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:55.409781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.412996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.442521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 361/715] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:55.452868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.467149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.484039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2a7a-258b-4b0a-aa0a-8c01ba944753 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.519129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.519761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.520039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.520310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.546023 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.548988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:55.575605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:55.576615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:55.577599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:55.577599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:55.577599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:55.580683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 355/716] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:55.587092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.660054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b959c4e-2c76-41e8-943d-188e031dfd84 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.702713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.732753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.734321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.734808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.735311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.736266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:55.736779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.738586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.745459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 362/717] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:55.782169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60b21-4909-468c-ae4a-7fa8d7ae58d1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.822839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.823453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.823736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.824012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.824667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:55.824992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.826362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc419b-1d2c-4971-9e3e-ce96682dce37 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.828976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.839852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 356/718] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:55.846045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.981740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe26f-0fc6-4d7d-aa92-259a2de39b31 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:56.022512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:56.023057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.023340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.023616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.024263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:56.025733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b75e2-10a3-4f0c-ad67-cf9d0ff2bc98 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:56.053333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 363/719] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:55 2022] POST /identity/v3/users => generated 254 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:56.917436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97daf-5257-4923-a96e-e74f6553aaae None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.917983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97daf-5257-4923-a96e-e74f6553aaae None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.917983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97daf-5257-4923-a96e-e74f6553aaae None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.919590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 357/720] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:56.925483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69bcf3-ca23-479a-9104-1cd42debb249 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.925906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69bcf3-ca23-479a-9104-1cd42debb249 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.926005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69bcf3-ca23-479a-9104-1cd42debb249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.949940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ae69bcf3-ca23-479a-9104-1cd42debb249 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:56.951142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ae69bcf3-ca23-479a-9104-1cd42debb249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:57.002298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.051895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 364/721] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:57.373171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5fd1-3dc8-4927-a81f-c9c2461b8409 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.373579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5fd1-3dc8-4927-a81f-c9c2461b8409 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.373780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5fd1-3dc8-4927-a81f-c9c2461b8409 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.375338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 358/722] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:57.380497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa109a-97b1-440e-8d22-198176696ebe None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.380788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa109a-97b1-440e-8d22-198176696ebe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.381090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa109a-97b1-440e-8d22-198176696ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.399476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c1fa109a-97b1-440e-8d22-198176696ebe None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:57.400648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c1fa109a-97b1-440e-8d22-198176696ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:57.480055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 365/723] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:57.527364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5fd1-3dc8-4927-a81f-c9c2461b8409 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.633503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21652f-8c62-4548-adbc-e788e0d9faea None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.633503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21652f-8c62-4548-adbc-e788e0d9faea None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.633503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21652f-8c62-4548-adbc-e788e0d9faea None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.634193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 366/724] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:57.639730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.640258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.640733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.659548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:57.661167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:57.697874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5fd1-3dc8-4927-a81f-c9c2461b8409 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.737235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 367/725] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:57.748153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.749367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.749653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.749931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.760705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:57.766335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:57.767269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:57.768113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 359/726] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:57.775184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.811838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.837448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.887137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c776d-3230-4f38-a6e1-9e3f3b2a7bc3 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.901655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.923991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.924608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.924893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.925216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:57.926408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.927552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.936782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 368/727] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:57.942234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.966975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e7d79-199e-4722-a05b-e69a3f03d12a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.032664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.033690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.034180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.034661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.035626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.037219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.046574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.052551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2c1b5-8c29-4976-9150-fc7b96573417 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.074691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 360/728] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:33:57 2022] POST /identity/v3/services => generated 240 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:33:58.116989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.117647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.117924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.118192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.130613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.132095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:58.148183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:58.149230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:58.150142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:58.150142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:58.150142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:58.150142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 369/729] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:58.155415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.210196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.324649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385846e1-5924-475f-a8a8-b03385de8f5f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.342611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.364995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.365619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.365904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.366182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.366827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:58.367130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.368486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c996f-0464-47dd-ab2c-7cad2169fa77 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.369483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56f600-a8b7-4d11-82c6-ccf04679cac7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.382530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 361/730] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:58.383628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a30ebc-45e7-494f-bd50-38ba8f25416d None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.383912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a30ebc-45e7-494f-bd50-38ba8f25416d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.384185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a30ebc-45e7-494f-bd50-38ba8f25416d None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.390071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 362/731] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:58.390850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a30ebc-45e7-494f-bd50-38ba8f25416d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.415731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.416675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.417221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.418294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.419513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.422317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a44268-9daf-4c6e-b437-1582bc840dc0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.437954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.445567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 370/732] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:33:58 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:33:58.450793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.450793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.451053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.492765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:58.496548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:58.597296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 371/733] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:33:58.614721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a30ebc-45e7-494f-bd50-38ba8f25416d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.674910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.675471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.676101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.676379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.684250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.685716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:33:58.698919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:58.700102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:58.701247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 363/734] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:58.706610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.817961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f04d-7659-4bb5-ad4b-1b29760bcfbd None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.848800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.859444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.860146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.860477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.860753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.861672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:58.861977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.863713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f683d3-12fc-4286-8ff4-ff12705f84da None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.886325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 372/735] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:58.890539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.892373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8a553-64e7-4724-afd4-4ec48246bb71 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.892664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8a553-64e7-4724-afd4-4ec48246bb71 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.892940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8a553-64e7-4724-afd4-4ec48246bb71 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.894536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 373/736] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:58.900499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.900780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.901082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.923569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:58.924765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:58.935535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.004370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 374/737] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:33:59.005440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.029202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aeaf-0b40-491d-9080-8dd063890203 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.067228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.067848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.068130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.074844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.167080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.168541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.213552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.280545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa01de7-d31a-4e99-801e-eabf1130538b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.300092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.324753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 364/738] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:33:59.331083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.331846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.332500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.332784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.333385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.354283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:59.361813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:59.363688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 375/739] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:59.365305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.463086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a61cb5-5c23-48ae-81f2-c9696e933a01 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.480889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8694-792c-485d-bc43-88246d0e31bf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.501179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.501950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.502328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.502702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.505564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.516314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.517895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.518080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.518608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.518888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.519163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.529946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:59.532984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:59.535129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:59.537396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:59.537396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:59.537396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:59.537800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:59.539793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 376/740] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:59.550035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.551800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 365/741] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:59.551800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.577983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.648909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93e45-9a88-4578-b7ec-001524f2543d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.653603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f019bb6-6a0b-4ca0-a1ba-b54ec29f9cc4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.687049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.688220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.688507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.688785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.689474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:59.689778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.691130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.691894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.692502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.692779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.693079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.693738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:59.694041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.695390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.708804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 377/743] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:59.709108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 366/743] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:59.709779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.710017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.779141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.817918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.826444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c916b9-769b-4b6e-aa05-97314941e242 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.849526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8af0f-aaa4-4753-9c60-9d299af3d2ac None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.869978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.870883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.871170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.871448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.872400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:59.872710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.875090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.887004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 378/744] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:59.888603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.892473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.893209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.893492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.893767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.905296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.907033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:33:59.927952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:59.929701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:33:59.931309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:33:59.931309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:33:59.931309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:33:59.934529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 367/745] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:59.936439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.014928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee460718-4b0c-4ade-8095-bf92aba66646 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.048001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.051209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f37c6-f7c0-4360-92ae-be047f46e7de None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.059396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.060134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.060511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.060813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.075168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.077160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.090477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.091380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.091854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.092831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:00.095011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.095873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:00.096762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:00.101379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 379/746] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:00.102229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.112184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.114434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.124850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:00.133449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.134394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:00.135270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:00.135270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:00.135270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:00.137590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 368/747] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:00.138673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.226768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f0df7-5389-4a92-b158-c91fd6a84f5a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.271803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.272565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.272982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.273361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.274101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.274492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.275959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.287005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da54a-5de9-491b-88a1-ede9c12f6d48 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.293203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 380/748] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.293203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.333332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.334038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.334435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.334782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.335575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.335977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.337705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.355971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 369/749] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.357031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.372746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.400318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.425990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d446855-7f58-4042-8e86-367c516d7ed3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.461974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.462593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.462872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.463150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.463790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.464097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.465501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.470747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c57eb-c2a6-49fd-8f19-e02d63bcceb1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.477380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 381/750] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.478192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.511001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.512016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.512636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.513155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.521460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.523269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-63c1cb16-ed99-4afb-9b6e-1b0599cec3e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:00.536471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.537474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:00.538681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:00.541020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 370/751] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:00.542178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ac78-3398-46c5-88a3-541e83492ac2 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.542470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ac78-3398-46c5-88a3-541e83492ac2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.542748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ac78-3398-46c5-88a3-541e83492ac2 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.544109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 371/752] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.548582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ac78-3398-46c5-88a3-541e83492ac2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.585668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3923b-39c8-44a2-ae40-4dfc89d3d484 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.621483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.622096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.622376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.622648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.623396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=24659ce97c204823a8968c757ea40e56) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.623703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.625159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.627145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.640986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 382/753] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/role_assignments?user.id=24659ce97c204823a8968c757ea40e56&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.640986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.644544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.682483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ac78-3398-46c5-88a3-541e83492ac2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.728907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.729698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.730053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.730401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.739589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.741203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-b2c3f222-e2cd-4bc0-a174-2cbfba632e89 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:00.754806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:00.755812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:00.756753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 372/754] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:00.758075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49ea0-2f0f-461d-b026-6509e9360bc8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.758507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49ea0-2f0f-461d-b026-6509e9360bc8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.758932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49ea0-2f0f-461d-b026-6509e9360bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.779534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f5b49ea0-2f0f-461d-b026-6509e9360bc8 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.781048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f5b49ea0-2f0f-461d-b026-6509e9360bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:00.781506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993f1d-9030-4d2b-814c-d87ce2c7c587 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.830277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.830929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.831209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.831488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.832223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.832530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.834028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.854396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 383/755] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.855175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.871947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 373/756] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:00.874923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bef2d-72a6-459b-9535-07aff18173a6 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.875347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bef2d-72a6-459b-9535-07aff18173a6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.875773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bef2d-72a6-459b-9535-07aff18173a6 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.877956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 374/757] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.881737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bef2d-72a6-459b-9535-07aff18173a6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.932249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.966280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.984553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893178-c2fd-4b6e-b36f-1f24833aaba0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.027297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bef2d-72a6-459b-9535-07aff18173a6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.029753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.030376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.030645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.030921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.031655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:01.031963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.033435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955987e8-d072-4de7-818e-1b01e43e74b1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.051794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 384/758] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.053153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.053444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.053715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.075490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:01.077246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:01.087205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.088555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.089111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.089640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.105004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.107261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:01.128368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.129416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:01.130345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:01.130345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:01.130345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:01.131316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 375/759] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.132302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.159871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 385/760] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:01.160724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.224923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.253369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2bd97-8199-4b54-98c8-c2e997ea1d37 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.287509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.288027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.288314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.288593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.293363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.296206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8482798-fc0c-457c-9915-997da0d82d4c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.347722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.348676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.349728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.350232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.351352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:01.352112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.354451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.366895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.368110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.371833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 386/761] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.372886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.488989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 376/762] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:01.488989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.492408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8a29-e657-4ecf-bc55-39b46e0978fa None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.502789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.503883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.530600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.531239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.531526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.531799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] PATH_INFO: `/v3/services/network` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.532379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.534208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.543617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.544610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.546154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 387/763] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.546938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.604886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cabf48-8655-491e-a47a-689eb4a0cae4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.644795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.645702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.646017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.646291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.652655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ee779-00a4-4e4b-8bd8-c0344febfca6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.653848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.655844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:01.667738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.668671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:01.669928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 377/764] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.670989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.697590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.698231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.698509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.698864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.699352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.700730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.710008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.711010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:01.711857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 388/765] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.712645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.753123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.766073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.792705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d5562-5216-4b87-aa84-dfd16d38781c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.824565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e5a5-df9c-43a9-b85c-58b300d4d9dc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.827813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.828442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.828720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.829017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.839891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.841362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:01.856336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.857502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:01.859098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:01.859098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:01.859098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:01.859098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 378/766] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.859325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.867821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.868484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.868818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.869142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.869767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] RBAC: Adding query filter params (name=network) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:01.870121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.871578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.876769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 389/767] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.877664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.969450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9447eb-ce58-48fd-87ac-31c05f921f67 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.977840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.981603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdaf4bb-7502-4aa3-abc3-81b6228d808d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.999306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.010860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.011706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.012060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.012683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.013767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.014757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.016487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.017724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.018401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.018748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.019089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.019794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.020165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.021619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.032008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 390/768] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.032008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.040660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 379/769] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.040660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.138197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ff0dc-f514-460b-b76c-ea3edb3f7a57 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.155287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfaeee-d3d8-478b-a9ed-3a96acf3060f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.174395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.175109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.175388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.175665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.176394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.176695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.178161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.194263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 391/770] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.194263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.197906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.198887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.198887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.199084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.199681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] RBAC: Adding query filter params (type=network) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.199984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.201901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.207591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 380/771] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.212971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.223002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.267164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.304400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ad96c-c2fd-4662-a86a-1dab7ae11085 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.320471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d0151-9814-4fa1-8644-61814ec8bd20 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.341732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.342969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.343250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.343521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.344199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.344498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.346224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.351832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 392/772] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.357263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.363090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.363853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.364233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.364614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.366740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=ad5260213b9f4ec49f3ce2ec48945d7d, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.368253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.381818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 381/773] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:02 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:02.382816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.501740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.502759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e761-cf87-4aae-aedc-5f21995a8e56 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.516882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.520589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d529751-3328-4774-8e29-d062bef2253a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.556045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.556662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.556940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.557244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.558599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] RBAC: Adding query filter params (interface=public, service_id=ddd50ac2f463428cb7d514762856a643, region_id=RegionOne) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.558599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.560280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.568655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 382/774] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/endpoints?service_id=ddd50ac2f463428cb7d514762856a643&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.569536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.570385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.571425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.571745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.572079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.579695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.581456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:02.597292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:02.598613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 393/775] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:02.599813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.711337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6cff0-1ec8-4c43-a0a5-e2511789277e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.750991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.751636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.751990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.752201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.752804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.754280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c772ca-0da6-4c6e-8024-330cedd88569 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.760506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 394/776] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.761858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728adef-a8ce-4d2d-9b48-f0c10ab160d0 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.762192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728adef-a8ce-4d2d-9b48-f0c10ab160d0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.762467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728adef-a8ce-4d2d-9b48-f0c10ab160d0 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.764573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 395/777] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:02.765501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728adef-a8ce-4d2d-9b48-f0c10ab160d0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.795276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.801462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ab7ff-216a-4068-b49e-d6f7c2aa85bf None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.845166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.845891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.846289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.846635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.847485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] RBAC: Adding query filter params (interface=public, service_id=86feb79dc40f45ad9d9a8b61faf64e45, region_id=RegionOne) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.848203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.849767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27d39-bbda-4c97-b9d3-06351379abb7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.860738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 383/778] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/endpoints?service_id=86feb79dc40f45ad9d9a8b61faf64e45&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.860738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.860738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.860738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.878382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:02.879816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:02.906162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728adef-a8ce-4d2d-9b48-f0c10ab160d0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.914492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.949152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.950016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.950298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.950582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.952662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.952987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.954564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.959173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 384/779] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:02.960196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.974478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 396/780] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.982048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.085690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.104644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7ef54-9bc7-469b-a337-a03d33542b94 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.130321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0223700-85f3-4d8f-965b-1157851b5d82 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.158034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.158567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.158819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.159095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.159807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.169107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.169107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 385/781] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.212833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.213750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.214187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.214642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.215546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=4128d9697d5e4f4b81eb3b46ec7b95b0) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:03.216015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.218272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.225413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 397/782] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/role_assignments?user.id=4128d9697d5e4f4b81eb3b46ec7b95b0&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:03.300901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.339892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.422689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d01e5-32b3-4611-af1d-ad6e2ff8f979 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.461942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.462588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.462869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.463144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.475510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:03.480796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:03.481765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:03.482668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 386/783] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:03.489364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.624119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f1153-db33-424f-9169-aaf6b7b0e3aa None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.664316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.665226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.665682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.666117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.666924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:03.667378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.668920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.671499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.681286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 398/784] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.684122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.797015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.810322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81cfe-8452-414f-b6cd-019daab35d27 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.857117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.858217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.858754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.859230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.875717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.877472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:03.897482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:03.898513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:03.899460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:03.899460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:03.899460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:03.901581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 387/785] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:03.907011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.012913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852030-b85b-49c3-b2c8-90f93807a6ba None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.051186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.052248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.052736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.053242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.054125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:04.054642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.056225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.069687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 399/786] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:04.076936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.235186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30abc2-451c-49e3-86c7-05176f1d9774 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.281378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.282011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.282292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.282561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.290836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.292303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5c4b04b5-83c9-486d-b426-2016381f26c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:04.306419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:04.307466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:04.308430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 388/787] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:04.314205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.396280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:04.451797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46deb8-9794-48b8-adee-f7402e511259 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.489392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dd0e-83c7-4bed-889f-eb177d50324b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.489588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dd0e-83c7-4bed-889f-eb177d50324b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.489871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2dd0e-83c7-4bed-889f-eb177d50324b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.493067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 389/788] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:04.499435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.499716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.499846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.500032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.500332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.500610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.500881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.501638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:04.501941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.503435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.515150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:04.534521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:04.535727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:04.539997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 400/789] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:04.550771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.678058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 390/790] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:04.711835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c350144-56e9-4519-83b9-e894ff62d902 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.730345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:04.758269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.758827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.759109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.759386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.771594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.773085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:04.789475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:04.790453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:04.791356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:04.791356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:04.791356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:04.791356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 401/791] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:04.797991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.919666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a989a9-7db4-470f-9f12-c2a1241eeafe None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:04.956611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:04.957363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:04.957648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:04.957925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:04.958631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:04.958940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:04.960293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:04.961902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:04.974171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 391/792] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:04.982066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:04.992421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.138600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef0636-a0c6-4fb4-83fc-391ff66da35a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.159015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5fe7d-ad40-4718-a445-a16262f402da None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.180083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.180729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.181086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.181370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.193175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.193940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.194346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.194716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.202575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.204142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.210361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:05.215631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:05.216580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:05.218033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:05.222065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 392/793] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:05.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:05.226745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:05.229199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:05.230184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.230327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 402/794] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:05.236129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.269335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.343878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.351212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d75290-2ca1-4732-b8cc-89432b24d985 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.395692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.396308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.396589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.396862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.397601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:05.397906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.399344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf71e7-c670-4c57-aa42-5825f1a12e05 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.403525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944db2-d9db-4700-9e63-bc6967f3a2eb None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.413084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 403/795] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:05.414524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4987a-96d1-4c2c-acfd-de09823383f7 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.414814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4987a-96d1-4c2c-acfd-de09823383f7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.415090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4987a-96d1-4c2c-acfd-de09823383f7 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.416541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 404/796] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:05.419627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4987a-96d1-4c2c-acfd-de09823383f7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.453967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.454598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.454879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.455152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.456106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:05.456487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.457873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc45dd-e316-43a0-99d0-85b15ca221d1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.474676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 393/797] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:05.476608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.476897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.477196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.501546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:05.505190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:05.521372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.541789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4987a-96d1-4c2c-acfd-de09823383f7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.590489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 394/798] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:05.590489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.595941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.596558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.599135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.599421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/24659ce97c204823a8968c757ea40e56/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.602075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=24659ce97c204823a8968c757ea40e56, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.605931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e51761-7c82-4339-b9d7-2d85fbeb5715 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.626076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 405/799] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:05 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/24659ce97c204823a8968c757ea40e56/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:05.668016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:05.742861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac9a45-6376-4d9a-9b74-5cb5bbd43b20 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:05.789003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:05.789662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.789945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.790219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.804871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:05.806364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:05.820761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4e25-19a2-4f4c-8a3a-dec30d298da2 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.822034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4e25-19a2-4f4c-8a3a-dec30d298da2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.822332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4e25-19a2-4f4c-8a3a-dec30d298da2 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.824360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 406/800] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:05.828726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:05.829830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:05.830748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:05.830748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:05.830748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:05.830919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7addf-532e-4f74-b7dc-073284268d63 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.831000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 395/801] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:05.831077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7addf-532e-4f74-b7dc-073284268d63 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.831077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7addf-532e-4f74-b7dc-073284268d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.835663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.883188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b2d7addf-532e-4f74-b7dc-073284268d63 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:05.884355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b2d7addf-532e-4f74-b7dc-073284268d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:05.966432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 407/802] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:05.968897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015da0-6e48-40a9-a166-99f4ead402d1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:05.969291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015da0-6e48-40a9-a166-99f4ead402d1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:05.969644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015da0-6e48-40a9-a166-99f4ead402d1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:05.975656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 408/803] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:05.975656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85015da0-6e48-40a9-a166-99f4ead402d1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:05.993513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.001484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7fc5c-f381-484c-8e15-453862acb135 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.044303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.045638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.046576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.047340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.048444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.048970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.051183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf28c71-74e0-4b78-b128-b1c19b104c81 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.075114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.075611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 396/804] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.078903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e988-dcdb-499e-9439-412efb794429 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.079778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e988-dcdb-499e-9439-412efb794429 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.080563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e988-dcdb-499e-9439-412efb794429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.101540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85015da0-6e48-40a9-a166-99f4ead402d1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.108244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8567e988-dcdb-499e-9439-412efb794429 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:06.110126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8567e988-dcdb-499e-9439-412efb794429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:06.139144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.139849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.140254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.140620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] PATH_INFO: `/v3/services/network` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.141355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.142887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.152251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.153483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:06.155139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.161486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 409/805] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:05 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.161486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.205131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 397/806] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:06.206065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e988-dcdb-499e-9439-412efb794429 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.285196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f3a4-ecbd-4d08-a2f7-83d3a4f13220 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.332547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.333486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.333913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.334980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.335829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] PATH_INFO: `/v3/users/nova` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.348443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.349074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e988-dcdb-499e-9439-412efb794429 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.350164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:06.364568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.371026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 410/807] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.372038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.402536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.403523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.404055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.404555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.405406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] RBAC: Adding query filter params (name=network) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.405991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.407682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.417057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 398/808] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.418083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.495406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be99ea6-dcb0-44a6-9802-5dbe4592dbed None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.532130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.533050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.533335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.533771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.534655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.536032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.536970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867aefa5-caed-4c1d-9e65-0692ca404f3a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.544885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.545984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:06.546862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 411/809] 10.222.0.25 () {56 vars in 1129 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.547694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.575000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.575926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.576964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.577264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.578125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.578432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.579847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.601755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 399/810] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.602556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.616074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.619028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.657811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282134f-50c4-4bf5-a942-0768691560ec None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.694558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.695234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.695517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.695785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.696655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] RBAC: Adding query filter params (type=network) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.696965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.698376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.708053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 412/811] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.708053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.714838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dc518-2f6e-4262-85b7-0341b849f46d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.752737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.753436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.753740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.754019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.765035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:06.769840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:06.770775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:06.771621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 400/812] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:06.772333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.822179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa20df-aee5-47df-bb9b-cd9b6adf3a57 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.851797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.854101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:06.861319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.861943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.862219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.862494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.863118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:06.863417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:06.865038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:06.873004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 413/813] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:06.873004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:06.895477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b698c-373b-4c68-880c-d3f7242862b0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:06.931439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:06.932152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.932432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.932703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.975535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82c381-05b4-4ba9-aa41-4f98093364e0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.003210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.004759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.008045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.008644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.008923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.009250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.009896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.011231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.041175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 414/814] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:06 2022] POST /identity/v3/endpoints => generated 337 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:07.042161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.083059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.111162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.122715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 401/815] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:07.124482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.180032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dea177-19c1-40e3-9042-23d8464226c1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.218204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.219189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.219473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.219744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.220819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.221434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.223060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.234953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 415/816] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.236517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.255136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc63ca-b9ae-4cfc-81db-f13fdddb06ea None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.298937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.299552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.299837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.300463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.300741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.301080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.302434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.309149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 402/817] 10.222.0.25 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.309149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.347234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.353522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80613592-e76e-4adc-aaa1-6e3418b09973 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.363326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.390052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.390676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.390954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.391227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.403005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.404524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.418281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117862-f1f7-4eb4-84a7-4a1063e65d20 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:07.418946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:07.419849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:07.421300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:07.421300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.421300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:07.421300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 416/818] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:07.421544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.461489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.462177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.462456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.462730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.476680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.478130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:07.494481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:07.495429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:07.496322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:07.496322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.496322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:07.496322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 403/819] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:07.496974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.530581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde14b-0b97-4971-a2fe-f822403754d1 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.566066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.566678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.566963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.567225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.567898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.569308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.586581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.590050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 417/820] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:07.601498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.608641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5027-3324-45c8-8d16-7ba3c4a3d41c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.623877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.670017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.671248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.671745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.672230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.673176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.673783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.675564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.690615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 404/821] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.698748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.730145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e59c94-0208-485e-895e-5b0164973ca7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.770073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.770960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.771241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.771516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.772165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.772473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.773879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89580-a551-4720-b550-60331a6e000d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.786892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 418/822] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.788123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2602d7e-dfee-45c0-a163-ffa949f04cae None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.788410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2602d7e-dfee-45c0-a163-ffa949f04cae None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.788744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2602d7e-dfee-45c0-a163-ffa949f04cae None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.790793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 419/823] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:07.795441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2602d7e-dfee-45c0-a163-ffa949f04cae None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.816490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cd207-1876-4b95-b597-bbff300d3297 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.854970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.855878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.856151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.856423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.863729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.865173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.876481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-cbeafe59-d71f-48f7-ab8c-95e43abbe7f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:07.879055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:07.880045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:07.881036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:07.885110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 405/824] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:07.886336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.886621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.886898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.893280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.905888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:07.906470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2602d7e-dfee-45c0-a163-ffa949f04cae None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.907055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:07.952352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.953007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.953284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.953553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.960779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.962242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:07.974055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:07.979669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 420/825] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:07.980943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.987419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 406/826] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:07.988313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.096167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f0ad-bfae-400c-bfa1-ba5322051810 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.106365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db755d-f8f2-49ff-a9b1-598d0b1984ee None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.133113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.134122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.134507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.135039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.136137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.136783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.138499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.138618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.142511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.147178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.148035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.148494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.149018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.150138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.151457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.153131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.154366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 421/827] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:08.159976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.176342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 407/828] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:08.181381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.270180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f7c0c-dfa5-49e1-82b7-8750dd2107bc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.305128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef0fa9-0ff3-4778-99b1-61c36039b471 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.306598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.307197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.308087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.308087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.308435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.308740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.310203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.317564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 422/829] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:08.322122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.347077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.347844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.348183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.348523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.360551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.362151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:08.378499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.379533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:08.380450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:08.380450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.380450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:08.380450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 408/830] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:08.389088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.402284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.451964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c535dd5-0d52-4a06-98c9-ba2f80452551 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.457353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.499074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.499707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.499982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.501497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.513892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:08.521600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.522614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:08.523523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 423/831] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:08.526617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447b4a-a322-4782-823c-48f464c62fef None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.528489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.565403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.566309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.566762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.567308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.568247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.568757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.570452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.585820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 409/832] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:08.591721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.663663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73087f-0ee2-4424-a1f9-13c9e73a9b59 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.690678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.706559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa552-bb88-4773-a586-43f6e49a25cc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.708313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.717509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.719085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.719636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.720477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.721662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.722171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.724821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.738347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 424/833] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:08.745017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.749521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.750333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.750753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.751192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.758548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.760156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:08.775001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.776086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:08.777196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:08.784596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 410/834] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:08.785659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.851838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0898d-1128-4123-9aad-c195919fa1f5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.887747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96230-c920-452b-8085-452a13b72f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.888573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96230-c920-452b-8085-452a13b72f73 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.889486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96230-c920-452b-8085-452a13b72f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.889872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96230-c920-452b-8085-452a13b72f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.894740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201fca0-681e-4865-a0ef-0c514f42ced7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.901417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def96230-c920-452b-8085-452a13b72f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.903060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def96230-c920-452b-8085-452a13b72f73 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-def96230-c920-452b-8085-452a13b72f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:08.919039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.920243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:08.921377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:08.921377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.921377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:08.927153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 425/835] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:08.928078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96230-c920-452b-8085-452a13b72f73 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.930665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.931380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.931664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.931965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.932722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.933053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.934470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.949060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 411/836] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:08.956603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.964937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.986473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.040608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96230-c920-452b-8085-452a13b72f73 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.079052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.079769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.080396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.080506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.081224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.081598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.083145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.087597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedb317-5704-45e0-9a0b-918b6b5eb7b9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.100028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 426/837] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.106135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.130545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.131287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.131484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.131762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/4128d9697d5e4f4b81eb3b46ec7b95b0/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.147267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=4128d9697d5e4f4b81eb3b46ec7b95b0, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.148822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.173545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 412/838] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:08 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/4128d9697d5e4f4b81eb3b46ec7b95b0/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:09.229134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.236747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba053d8-3b3c-4559-bf3b-26b8b3358416 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.289190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.290125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.290554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.291040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.299712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.301487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:09.315507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:09.316708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:09.317913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:09.320107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 427/839] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:09.326983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.401474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.433217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f36c4-2b2f-4e5b-aa63-74a7e68c20d3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.469753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.471042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.471469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.471888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.472846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.473312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.474894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df212e-8339-4246-92b9-96a58e3d7ed8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.493825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 413/840] 10.222.0.25 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.500002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.606315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465570-7c33-44b2-bbc5-d8d16813f5d8 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.606618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465570-7c33-44b2-bbc5-d8d16813f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.606841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465570-7c33-44b2-bbc5-d8d16813f5d8 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.609010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 414/841] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:09.613767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.614059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.614335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.626054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d5af5-0fde-4d6d-8d23-2b72ee4870ca None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.633914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:09.635111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:09.662439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.663306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.663781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.664323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.694496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.724309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 415/842] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:09.753698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.755430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.859727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 428/843] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:09.866080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.932143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.974287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b5399-3709-4376-adaf-355d14ceceed None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.999655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.014132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.014777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.015047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.015317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.026941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.028382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:10.047830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:10.048857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:10.049799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:10.049799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:10.049799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:10.051866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 416/844] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:10.057548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.068903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.129596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348fa20-9067-4297-8af8-19791a6bd169 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.173630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.174656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.175071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.175852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.176857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.178644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:10.190105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:10.191332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:10.192367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:10.195168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 429/845] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:10.200715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a3013-7461-409d-aa3c-678327f964b9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.200868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.215805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.239230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.240108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.240533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.240950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.241761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:10.242209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.243711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.261308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 417/846] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:10.262462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.318455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34246953-03a9-4ccb-8513-bbf8b58cf646 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.340005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.361375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.362125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.362514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.362871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.363598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:10.363996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.365723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.371393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac405c3f-7e99-4a86-9dfc-8608118414a4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.375346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 430/847] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:10.376314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.409398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.409982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.410239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.410513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.417249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.418682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-e33c12bb-7ad2-474a-b3b0-b455efee6793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:10.430634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:10.431530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:10.432439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 418/848] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:10.432927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644dd2-9d59-4aaf-bfcc-383a3e3b812f None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.433940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644dd2-9d59-4aaf-bfcc-383a3e3b812f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.433940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644dd2-9d59-4aaf-bfcc-383a3e3b812f None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.434743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 419/849] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:10.438119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96644dd2-9d59-4aaf-bfcc-383a3e3b812f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.474721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.487507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f0a43-94f8-4b19-82cc-6b71937a299e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.524703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.526606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.527345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.527836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.528584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:10.529051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.530582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d791c9b-ac2a-4b8b-813c-423b0a24487f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.535444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 431/850] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:10.537176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.537922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.538382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.549647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96644dd2-9d59-4aaf-bfcc-383a3e3b812f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.567525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:10.569418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:10.589295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.589930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.590284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.590566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.591338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:10.591653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.593259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.612615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 420/851] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:10.613654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.629259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.665069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 432/852] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:10.733503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.742400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0174006-50e2-489c-b382-f3d6979c048a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.778380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.778980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.779343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.779618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.780336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=24659ce97c204823a8968c757ea40e56) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:10.780639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.783035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98642f5b-d20b-49a3-b9c8-344ed661144f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.789453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 421/853] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/role_assignments?user.id=24659ce97c204823a8968c757ea40e56&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:10.968999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.194710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58942df4-9bda-4400-a3bb-5dd3c79f0e55 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.206860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2b64e-937a-458b-a68b-dc8ebd016913 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.207461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2b64e-937a-458b-a68b-dc8ebd016913 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.207962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2b64e-937a-458b-a68b-dc8ebd016913 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.209538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 422/854] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:11.214556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.214836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.215111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.233575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.234530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.234711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:11.235178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.235680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.236325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:11.236675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.238270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:11.250378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:11.251397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:11.252340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:11.254600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 433/855] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:11.260218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.324589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 423/856] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:11.325958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.385844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85945-3fba-4a17-8480-280856f5d759 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.422301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.423235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.423749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.424227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.425124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:11.425649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.427211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.432605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 434/857] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:11.439519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.527989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.543675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1802f1-ac92-44f7-a004-d9a58d2b79e3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.578973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.579581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.579856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.580124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.580742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:11.581069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.582419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.587256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 424/858] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:11.600828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.719672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adfba-64b8-4ff9-8f82-f28f84d3e059 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.722381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.756549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.757217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.757499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.757774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.768908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:11.773746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:11.774656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:11.775492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:11.780231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 435/859] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:11.781625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.885743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456fada-b7ab-4174-848d-3909c546f845 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.922016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.923001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.923478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.923944 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.925008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:11.925545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.927120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.938554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 425/860] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:11.946894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.061978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7994313-6158-43ca-8a60-91f1dea8c03f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.100213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.100899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.101198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.101473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.111994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.113472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:12.132210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:12.133224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:12.134139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:12.134139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:12.134139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:12.139622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 436/861] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:12.143059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.169318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.248662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef30a-e515-4624-aa6e-e2c37027c96b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.284007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.284623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.284980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.285263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.285967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:12.286281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.287653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.308683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 426/862] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:12.311182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.333604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.423999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13195be4-cd9e-45c7-8a45-d78959048f81 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.459324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.460067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.460353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.460680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.471146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.473964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-70ce3bda-e1be-4301-af63-0d1c426d4a22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:12.486749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:12.487695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:12.488630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:12.492577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 437/863] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:12.493991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.548465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac57e-5e0b-498b-a59d-2071862f195b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.548786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac57e-5e0b-498b-a59d-2071862f195b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.549032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac57e-5e0b-498b-a59d-2071862f195b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.550635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 438/864] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:12.556785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dee97-97ae-4f2d-b39d-c268ee5f8440 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.557202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dee97-97ae-4f2d-b39d-c268ee5f8440 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.557597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dee97-97ae-4f2d-b39d-c268ee5f8440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.574206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.581177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-fe5dee97-97ae-4f2d-b39d-c268ee5f8440 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:12.582521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fe5dee97-97ae-4f2d-b39d-c268ee5f8440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:12.608756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80418c81-aac8-48dc-baa3-d36ef0b7347f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.646764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.647652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.648097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.648518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.649562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:12.650116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.651827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.673946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 427/865] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:12.680481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 439/866] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:12.686667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.790409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e163d-ede8-4139-b2a9-92fb5bee32e9 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.790741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e163d-ede8-4139-b2a9-92fb5bee32e9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.790951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e163d-ede8-4139-b2a9-92fb5bee32e9 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.796686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 440/867] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:12.800483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.800764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.801065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.801792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9b7b-cb9c-4d05-aa01-766547463a64 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.818584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.824468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:12.826136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:12.839323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.848759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.849642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.850104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.850557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.913576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 441/868] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:12.937331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.938798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.950834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.068917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81293050-9e79-49f2-833a-f66aba0b36aa None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.070122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 428/869] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:13.078612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.111665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.115473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.116846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.117367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.118810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.119634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.121176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.126639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.145130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 442/870] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:12 2022] POST /identity/v3/services => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:13.198947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.215232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fe24-8934-4f0f-ab2b-bb78fe396ed4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.270870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.271767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.272934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.273387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.291042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.293020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:13.312179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.313393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:13.314356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:13.314356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:13.314356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:13.314356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 429/871] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:13.319914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.343307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced95c4-c6d8-4271-bc5b-0d42ab550213 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.380546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.381327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.381610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.381883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.382461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.383943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.394254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.395270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:13.396119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:13.400539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 443/872] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:13.401748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.423950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.452494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a417e1-3124-4bf0-a95d-c66b3282a82d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.481297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.495386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.495996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.496278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.496553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.497220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.497525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.499438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.512977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 430/873] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:13.514838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7d65-4c8a-4360-a18d-24cdfb59cacf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.519150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.557030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.557811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.558100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.558377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.559006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.559623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.561353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.571384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 444/874] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:13.572651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.657684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fab8e-9226-47a4-9818-87343cc29a7f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.684297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050849c2-7cdd-43fe-9971-58c0a20bb660 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.699990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.701630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.702194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.702474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.710412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.710898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.711728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.714210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.723316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.723922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.724198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.724467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-70ece569-499f-49c2-8167-ef9b72315b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:13.724786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.725836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:13.731719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:13.732142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.732142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.732328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 431/875] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:13.732398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a76c6-e09d-47ff-9e08-35887c437b48 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.732464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15de33-0dcd-4656-b8b7-b52286887a7b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.732464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15de33-0dcd-4656-b8b7-b52286887a7b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.732464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15de33-0dcd-4656-b8b7-b52286887a7b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.732464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 432/876] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:13.732464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15de33-0dcd-4656-b8b7-b52286887a7b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.737435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 445/877] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:13.738510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.738790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.739058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.765544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:13.767102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:13.863094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15de33-0dcd-4656-b8b7-b52286887a7b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.873541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 446/878] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:13.908239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.908850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.909184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.909463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.910411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.910517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.911860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.924653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 433/879] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:13.930867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.039953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0bd1-a704-400a-9412-3f3729d8b368 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.058012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.078884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.079490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.079768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.080039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.080761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=4128d9697d5e4f4b81eb3b46ec7b95b0) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.081093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.082523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b950d7e-5d92-475f-a8a2-9b475e29734d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.088861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 447/880] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/role_assignments?user.id=4128d9697d5e4f4b81eb3b46ec7b95b0&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.155077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.229679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.274158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ab97d-3ffc-4f91-a541-187823565479 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.309963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.310526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.310806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.311079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.311725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.313116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde745c3-4143-47b3-943d-797e82916ba4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.332652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 434/881] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:14 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:15.199473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba9ea-dff9-494e-9cbf-0f9170df4456 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.200483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba9ea-dff9-494e-9cbf-0f9170df4456 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.201588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eba9ea-dff9-494e-9cbf-0f9170df4456 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.203411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 448/882] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:15.208840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d3bf-41b6-4cf3-ac1c-ea2518d54014 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.209706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d3bf-41b6-4cf3-ac1c-ea2518d54014 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.210190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d3bf-41b6-4cf3-ac1c-ea2518d54014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.228670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d146d3bf-41b6-4cf3-ac1c-ea2518d54014 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:15.230199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d146d3bf-41b6-4cf3-ac1c-ea2518d54014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:15.312526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 435/883] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:15.586062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eba9ea-dff9-494e-9cbf-0f9170df4456 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.699876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eba9ea-dff9-494e-9cbf-0f9170df4456 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.736557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.737222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.737505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.737778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.738358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.739724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-c095a599-5710-45f1-aaea-402fd772fa2c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:15.749847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:15.750934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:15.751809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:15.752558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc42fa-e5c9-4efa-a082-788488a2afcc None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.752558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc42fa-e5c9-4efa-a082-788488a2afcc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.752558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc42fa-e5c9-4efa-a082-788488a2afcc None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.752755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 449/884] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:15.752826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 436/885] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:15.757679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.757966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.758243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.759965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc42fa-e5c9-4efa-a082-788488a2afcc None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.776474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:15.777688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:15.813214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.833315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.861303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 450/886] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:15.874175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc42fa-e5c9-4efa-a082-788488a2afcc None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.910151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.910753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.911030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.911298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.911915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:15.912213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.913588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.918800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 437/887] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:15.924995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.030325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c7c65-4190-4002-b28c-cac89a394fa0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.066117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.067076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.067561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.068055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.068986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.069496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.071351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dd6bd-1bcf-4c89-922e-3e46746d618e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.077128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 451/888] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:16.083265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.157787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cde8f-1b1a-451e-b139-d808a54c28a5 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.158458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cde8f-1b1a-451e-b139-d808a54c28a5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.158936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cde8f-1b1a-451e-b139-d808a54c28a5 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.160442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 452/889] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:16.161461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cde8f-1b1a-451e-b139-d808a54c28a5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.190309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf0dd-4ca9-4fb6-9c3c-cc3d1adc7f56 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.201906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.207023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.232289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.232893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.233196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.233572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.234164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] RBAC: Adding query filter params (interface=public, service_id=883036dd778c4246b2dc1c131eb7e4f7, region_id=RegionOne) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.234471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.235822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32ed30-4938-4841-bc40-53541382192c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.245391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 438/890] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/endpoints?service_id=883036dd778c4246b2dc1c131eb7e4f7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:16.245391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609295e-37e5-4677-87b1-88f7f548a65a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.245391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609295e-37e5-4677-87b1-88f7f548a65a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.245391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609295e-37e5-4677-87b1-88f7f548a65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.262391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0609295e-37e5-4677-87b1-88f7f548a65a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:16.264121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0609295e-37e5-4677-87b1-88f7f548a65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:16.274262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cde8f-1b1a-451e-b139-d808a54c28a5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.313246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.314071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.314354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.314627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.315333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.316787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.333548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 453/891] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:16 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:16.342790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.345451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 439/892] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:16.350105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2d1e4-8c02-4f73-8c42-ba39b4035a54 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.350395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2d1e4-8c02-4f73-8c42-ba39b4035a54 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.350674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2d1e4-8c02-4f73-8c42-ba39b4035a54 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.352554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 440/893] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:16.359279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.359977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.359977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.389196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:16.390839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:16.480715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.486771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ffe42-4204-4604-af21-0b505c00f334 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.490540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 441/894] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:16.536667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.537344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.537626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.537905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.538513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.539900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.549046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 454/895] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:34:16.618421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.649830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.734762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ba668-de78-4dcc-ae50-720ee8a2ac04 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.774256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.774900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.775179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.775450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.782949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.786322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:16.793025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.794003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:16.794894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:16.795627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 442/896] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:16.800542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.908537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.922175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a243c-cd8a-4e4d-8e38-9faf1386aa52 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.922913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbfb28-6e20-4c5c-afb9-945fe357d5cc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.960015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.961085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.961574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.962106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.962887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.964533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.966133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.967116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.967465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.967807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.968943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.969357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.971153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.975493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.976664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:16.977812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:16.980375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 455/897] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:16.982782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 443/898] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:16.989145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.989903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.020531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.120639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91dfbf-cf41-4666-8f81-ee436d5866e8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.122179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f843-a9b5-4a6e-9f76-4f3b55188b9d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.164498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.165360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.165793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.165994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.166276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.166599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.166872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.167341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.167564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.167726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.170205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.180804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 444/899] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.181538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.182720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.184504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.204200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:17.206124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.207072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:17.208500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:17.208500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.208500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:17.214067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 456/900] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.219104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.313510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.319973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17128d-3892-4a27-ba8c-023c731b44c4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.325496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2389e-80ca-42aa-8e95-091e7c6d370d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.368631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.369855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.370402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.371202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.372033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.372690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.374009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.374517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.374839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.375221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.375595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.376603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.377059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.379144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.382857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 445/901] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.389013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.397147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 457/902] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:17.398668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.474096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.498297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ee2b-7123-4938-a662-2f7706e1857f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.529990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd1ca7-bbbc-4851-962a-9b2a8a161972 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.537267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.537879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.538163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.538437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.539142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] RBAC: Adding query filter params (interface=public, service_id=3668e62ecaa5430399e766b5e4e39cf3, region_id=RegionOne) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.539448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.540804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.552531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 446/903] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/endpoints?service_id=3668e62ecaa5430399e766b5e4e39cf3&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:17.558219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.580759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.581348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.581628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.581899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.591359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.593557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.597472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:17.610278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.611281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:17.612232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:17.616620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 458/904] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:17.617571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.692429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83e7c1-9321-47c7-8c3f-453afe718b38 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.738060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.739252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.739717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.740331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.741464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.743390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.747095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fc5fa-581b-4ee9-ac0a-e5be9d25cd98 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.757569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 447/905] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:34:17.788428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.789141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.789426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.789702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.790447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.790755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.792389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.812637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 459/906] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.818498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.829634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.935000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16779b-ddb4-4520-b502-5a9ef33023a8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.974196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.975162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.975646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.976135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.051918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.053400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.062340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.163261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 448/907] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:18.168881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.230219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.279264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8383c91-d8d9-456f-be0d-5aff27831394 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.314756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.315453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.315781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.316063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.327574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.329134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-833676f9-d5c7-4c54-bc90-287de43bc4aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:18.344356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.345392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:18.346313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:18.346313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:18.346313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:18.346313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 460/908] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:18.352719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.374215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd4ea-57e8-4798-ab04-6d0d16173629 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.374503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd4ea-57e8-4798-ab04-6d0d16173629 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.374775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd4ea-57e8-4798-ab04-6d0d16173629 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.376350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 461/909] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:18.381300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.381583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.381854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.405859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:18.407105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:18.474098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cb6b0-cceb-4423-a412-8b8e1f03caac None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.483151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.483614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 462/910] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:18.512214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.513126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.513591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.514010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.514791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:18.515264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.516778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d205645-1e6c-4686-9d7e-2a010070c5a7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.529126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 449/911] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.536787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.569664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887026-4239-421e-bb48-6e456b645469 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.569860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887026-4239-421e-bb48-6e456b645469 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.570140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887026-4239-421e-bb48-6e456b645469 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.571958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 450/912] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:18.576758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.577090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.577386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.583183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.598203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:18.599415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:18.657999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75b111-a648-48cc-bc5e-61d12f00a2d1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.680655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 451/913] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:18.697989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.698928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.699358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.699777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.706792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.708488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:18.720061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.721181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.722335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:18.724328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 463/914] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:18.730044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.755465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.835228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.844899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb76557-3f92-42d7-9874-abcaeb743f48 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.862058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.865447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed495-655e-405a-b951-4ac6003e1683 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.894126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.894991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.895358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.895737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.896674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:18.897119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.898646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.909814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.910452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.910737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.911016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.911597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.912040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 452/915] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.913179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.917977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.923989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.925020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:18.925878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 464/916] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:18.930800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.037164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d83d7-ce52-4c3c-9b47-d36fdf4ea429 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.044724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b8da-876b-4f7e-8232-14896bb74cce None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.076478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.077416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.077855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.078120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.078908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] RBAC: Adding query filter params (role.id=f8aa2a4fa2864b8f9472faee4c3f0f33, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=4128d9697d5e4f4b81eb3b46ec7b95b0) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.079219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.080683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.081568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.082176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.082515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.082722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.083367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.086433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.087810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.097311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 453/917] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/role_assignments?user.id=4128d9697d5e4f4b81eb3b46ec7b95b0&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.105444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.105643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 465/918] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:19.106743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.146724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.149386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.231487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195176ee-0293-49e3-a9ad-abc2fbf2d56e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.236219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80885f-8f5e-453b-ba91-1f6f506bf6e0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.275592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.276209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.276491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.276764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.277429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.277544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.277730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.278446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.278937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.279198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.279505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.280219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.282232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.289356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 466/919] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:19.290245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:19.295825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:19.297041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:34:19.297949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:19.305478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 454/920] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:19.306769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.400383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.411678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d871437-4770-4097-a78a-5593d9e4b4d7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.412152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.429740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be469b-3acf-48d7-8561-7545bbc6486c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.458867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.459850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.460322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.460734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.461702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] RBAC: Adding query filter params (interface=public, service_id=b842a10ce220413aa94b7bf60a8696bb, region_id=RegionOne) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.462186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.463912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.464810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.465451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.465731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.466001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.466656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.466962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.468293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.473197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 467/921] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/endpoints?service_id=b842a10ce220413aa94b7bf60a8696bb&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.478173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 455/922] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.479043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.479191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.584362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e9ad-85ca-4260-b1e0-fb3a3b139556 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.589105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dba727-6015-4066-acff-396ac438f5ef None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.619709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.620314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.620592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.620864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.621588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.621824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.623207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.627976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.628824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.629454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.629925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.630726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.632358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15afa6-4f72-4bb6-93e8-120303f06209 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.633335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 456/923] 10.222.0.25 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.634605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.639942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 468/924] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:34:19.650457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.748359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5b8df-1aad-4595-b1b5-8f45e42c3677 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.788804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.789374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.789827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.790107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.790779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.792176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2431d01-d62d-4646-be98-8c1d398e4c47 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.816710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 457/925] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:19.840152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c43d0-a947-4742-bffe-d260b678e947 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.840405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c43d0-a947-4742-bffe-d260b678e947 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.840784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c43d0-a947-4742-bffe-d260b678e947 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.843237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 469/926] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:19.850284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.850617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.850850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.873766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:19.875348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:19.949189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.962802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 458/927] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:20.246394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c43d0-a947-4742-bffe-d260b678e947 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.270600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.360081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c43d0-a947-4742-bffe-d260b678e947 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.395022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.395649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.395930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.396201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.396773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.398175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:20.407071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:20.408115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:20.409138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 470/928] 10.222.0.25 () {56 vars in 1132 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:20.414226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.518364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6f51-4a36-4ae0-9afd-f632caec30cb None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.554744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.555363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.555643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.555919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.556552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.556857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.558251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.563994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 459/929] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.569603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.656283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.672682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf44fd-bed3-41c7-824a-165d7ba59a57 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.708675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.709319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.709600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.709872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.710497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.710801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.712167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadbf6a-7142-4be3-91b2-23d8f8c6c6fc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.717657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 471/930] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.723217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.785600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.831507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560c2b-073c-4019-93d5-5b7e41de84fc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.867277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.867891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.868163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.868439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.869144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.870504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c07ff3-c8d7-4b26-96b1-ba2ca9dd6b25 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.893992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 460/931] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:21.089820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099983b-8cbb-45bc-8076-b9e760580a2c None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.090348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099983b-8cbb-45bc-8076-b9e760580a2c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.090841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099983b-8cbb-45bc-8076-b9e760580a2c None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.093187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 472/932] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:21.098695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.099237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.099727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.119458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:21.120981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:21.195682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 461/933] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:21.471186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099983b-8cbb-45bc-8076-b9e760580a2c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.586006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099983b-8cbb-45bc-8076-b9e760580a2c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.624631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.625298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.625585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.625864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.636359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e7043b07-834a-495c-bc0c-8e2428e7e10d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:21.641778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:21.642717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:21.643570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 473/934] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:21.650646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.693218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.751539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bef1d-cc04-427d-8a1a-c15b4a5c1757 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.751736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bef1d-cc04-427d-8a1a-c15b4a5c1757 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.751999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bef1d-cc04-427d-8a1a-c15b4a5c1757 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.755096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 474/935] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:21.760022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd658d4-0910-463f-b6d9-6c2f5801bcd6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.760648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d0d4-1acb-4a05-a7f3-33c21965bd20 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.761221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d0d4-1acb-4a05-a7f3-33c21965bd20 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.761704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d0d4-1acb-4a05-a7f3-33c21965bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.781715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6c40d0d4-1acb-4a05-a7f3-33c21965bd20 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:21.783337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6c40d0d4-1acb-4a05-a7f3-33c21965bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:21.785799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.806478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.807124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.807404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.807676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.808782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:21.808782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.810686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec973bc-a2c1-40ea-874c-f5c5ed926992 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.820170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 462/936] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:21.826853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a80ea-f44a-4da8-a730-0cb5d29aa7f6 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.827140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a80ea-f44a-4da8-a730-0cb5d29aa7f6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.827405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a80ea-f44a-4da8-a730-0cb5d29aa7f6 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.828642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 463/937] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:21.829372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a80ea-f44a-4da8-a730-0cb5d29aa7f6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.859833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 475/938] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:21.861329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.861621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.861897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.881660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:21.883288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:21.959307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 476/939] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:21.960189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a80ea-f44a-4da8-a730-0cb5d29aa7f6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.961801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.994606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.995239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.995520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.995789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.006561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.007991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:22.023104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.024086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:22.025055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:22.025055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:22.025055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.025055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 464/940] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:22.030042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.105313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.135599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb2fc-e210-4a18-8e32-fcfeafe52bb3 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.157602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.170737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.171349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.171627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.171899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.172535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.172839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.174247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.186408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 477/941] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.191735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.268763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7308b3-aa39-4336-b9f8-cb2393ebe189 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.294918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb3729-56aa-4ff8-897e-df89bbd507aa None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.302714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.303566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.303850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.304129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.304722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.306112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.315684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.316763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.317669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 465/942] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:22.318492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.329579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.338727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.339366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.339662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.339930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.342294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.348124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.349686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:22.361604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.362583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:22.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:22.367519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 478/943] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:22.368365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.428877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085b179-00f8-486f-b035-9a37be6b8ab6 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.469465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.470081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.470360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.470636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.471276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.472689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.473891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864c33-935a-40c0-9f09-124414e55578 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.475342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING keystone.server.flask.common [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 09:34:22.495360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.sql.core [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:34:22.495360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:34:22.495360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [parameters: {'id': '984b7243814243df9d48ee47ebd3c303', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:34:22.495360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88241) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 984b7243814243df9d48ee47ebd3c303.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 984b7243814243df9d48ee47ebd3c303. Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:22.506368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 09:34:22.507199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application yield resource Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise self.value Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application session.commit() Aug 31 09:34:22.508283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:34:22.509229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:34:22.510169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:22.512374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [parameters: {'id': '984b7243814243df9d48ee47ebd3c303', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.513800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.514882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 984b7243814243df9d48ee47ebd3c303. Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 466/944] 10.222.0.25 () {60 vars in 1140 bytes} [Wed Aug 31 09:34:22 2022] POST /identity/v3/projects => generated 262 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:22.516170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.517104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.517104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.517104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.517104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.517462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.517462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.518098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.534914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 479/945] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.534914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.600193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.605095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.634942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d1c2f-fd23-4933-b1f1-84b1108fc124 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.654239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8e56c-627a-4f5b-a35a-063ef211b9be None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.671366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.671877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.672109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.672317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.673079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.673389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.674991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.693126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.693756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.694040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.694316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.695496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 467/946] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.697991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.702558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.704009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:22.715545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.717459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 480/947] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:22.718411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.808354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf50f4-1ac0-4376-9e0f-53249639cb6d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.833106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f922a1a-994c-423b-bb59-e72727cb1175 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.839412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.846476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.847411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.847879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.848133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.848770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.849114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.849718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.850776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.855599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 468/948] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.856359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.881267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.882048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.882431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.882831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.894895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.896335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:22.912710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.913931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:22.915672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:22.915672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:22.915672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:22.915672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 481/949] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:22.915672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.962133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f594d-26d7-4c77-8fc2-a2cf43e081fe None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.995830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.996427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.996704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.997000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.997709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.998009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.999411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54371695-6e5f-4f6e-b4eb-3ee94ebb4189 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.014628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f893d01-4ad4-4f34-bba8-bdc6332fb296 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.016762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 469/950] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.017848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00931-21c9-4258-8b4c-963f3007a406 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.018127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00931-21c9-4258-8b4c-963f3007a406 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.018396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00931-21c9-4258-8b4c-963f3007a406 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.019581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 470/951] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:23.020278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a00931-21c9-4258-8b4c-963f3007a406 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.065270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.066300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.066900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.067244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.067949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.069569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a15e32-59d0-41f7-8352-82e539cb8b0d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.099254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.102303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 482/952] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:23.113162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.113162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.113162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.128053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:23.129861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:23.152796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.181164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a00931-21c9-4258-8b4c-963f3007a406 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.228283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.228979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.229275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.229553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.230246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.230551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.231968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.235516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 483/953] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:23.245085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 471/954] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.251975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.357299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d042a25-8ed7-4c6d-a2d9-b59ce6c7be07 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.393131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.393775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.394054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.394325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.401379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.402837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:23.414300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.415354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:23.417061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 484/955] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.421307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.501296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.516201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.537730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2654b-3b16-4225-8aaa-2eee87627b2d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.551717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.577257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.578377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.578858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.579328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.580214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.580753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.582641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.597126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 472/956] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:23.603353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.637023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e3821-5df7-4fa8-975c-321413ebc884 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.673317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.674205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.674493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.674868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] PATH_INFO: `/v3/services/compute` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.675544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.677351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.687014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.688368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:23.689336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:23.690124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 485/957] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.695147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.720849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967566d-b20e-42e8-b25d-5da4167756e1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.759314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.760304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.760860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.761357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/4128d9697d5e4f4b81eb3b46ec7b95b0/roles/f8aa2a4fa2864b8f9472faee4c3f0f33` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.763995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=4128d9697d5e4f4b81eb3b46ec7b95b0, role_id=f8aa2a4fa2864b8f9472faee4c3f0f33)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.765736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.775617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.789110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 473/958] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:23 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/4128d9697d5e4f4b81eb3b46ec7b95b0/roles/f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:23.805542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330ae32-7d60-478d-bc67-b58511125e3f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.845537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.846156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.846434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.846704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.847332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] RBAC: Adding query filter params (name=compute) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.847630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.849022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebee24a-db0b-485f-aebc-ea0a2f123a4b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.854987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 486/959] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.861377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.896177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.972728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab94d-2853-44c9-b356-8eda07642cbc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.008906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.009553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.009832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.010103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.010725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] RBAC: Adding query filter params (type=compute) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.011024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.012365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeefe0-467f-4f71-a589-ecb3976ac21f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.017404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 474/960] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.121893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0d4b-1a0b-4ca9-9f27-c6a2e9486103 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.122451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0d4b-1a0b-4ca9-9f27-c6a2e9486103 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.122698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d0d4b-1a0b-4ca9-9f27-c6a2e9486103 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.124944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 487/961] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:25.130918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.131339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.131526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.154691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:25.156211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:25.159176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d108-7fc5-4365-a67b-6efd968e1e6c None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.159466 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d108-7fc5-4365-a67b-6efd968e1e6c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.159744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d108-7fc5-4365-a67b-6efd968e1e6c None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.161373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 488/962] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:25.166237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.166522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.166798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.187237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:25.189033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:25.230056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 475/963] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:25.272094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 489/964] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:25.512591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.541847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.546616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.623987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c3290-a7ce-486c-9df3-3784f304382d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.632203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.670042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.670659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.670938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.671213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.671863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.672491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874ded3-1910-47cd-94d5-df5199372121 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.673309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.689272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.sql.core [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:34:25.689272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:34:25.689272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [parameters: {'user_id': '11027dbc197c48faaa746a07cbedf4b2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:34:25.689272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88241) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:25.698198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:25.699337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application yield resource Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise self.value Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application session.commit() Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:34:25.701310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:34:25.702222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:25.703180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:25.704091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [parameters: {'user_id': '11027dbc197c48faaa746a07cbedf4b2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.705164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 09:34:25.706030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 476/965] 10.222.0.25 () {60 vars in 1131 bytes} [Wed Aug 31 09:34:25 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:25.707159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.726265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.727290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.728157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.728588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.729701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.731473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.744139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:25.745484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:25.746552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:25.750141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 490/966] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:25.756452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.816167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.832068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b009c8-c9e2-4604-bbdb-e70494498716 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.864203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90dd47-1009-4af5-854d-bbe7ad548de3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.869553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.870257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.870604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.871048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.879522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.881276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.894723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-30e74a96-40a6-443d-89a2-ec0d8c1473c3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:25.895414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:25.896526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 477/967] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3680a4-4a28-43c0-b3f2-380cff474724 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.902580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3680a4-4a28-43c0-b3f2-380cff474724 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.903444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3680a4-4a28-43c0-b3f2-380cff474724 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.906993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 478/968] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:25.907590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3680a4-4a28-43c0-b3f2-380cff474724 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.908381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.909133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.909517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.909891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.910625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:25.911073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.912506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b244-f720-44ee-b17d-e8f9d444deed None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.922900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 491/969] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.922900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.922900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.922900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.940156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:25.941452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:26.007921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 492/970] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:26.009021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.022678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3680a4-4a28-43c0-b3f2-380cff474724 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.058551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.059357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.059644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.060463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.061193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.061497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.063033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb57c1a-25d4-4724-8ce8-2842eb951783 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.105844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 479/971] 10.222.0.25 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.107894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623416-3e1c-4571-b5be-35f5110e6d78 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.108262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623416-3e1c-4571-b5be-35f5110e6d78 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.108556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15623416-3e1c-4571-b5be-35f5110e6d78 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.109857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 480/972] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:26.115391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.115676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.115952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.125531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6cb19-dad7-4cc2-be15-a58ecb6bd4ee None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.135891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.141630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:26.143353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:26.155554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.165012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.165641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.165920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.166199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.166839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.167147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.168638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.175183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 493/973] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.180923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.243448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 481/974] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:26.274201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.303133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64431ae-7850-4b05-9a04-b74cf4e76a42 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.344082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.344800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.345159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.345778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.346762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] RBAC: Adding query filter params (interface=admin, service_id=b842a10ce220413aa94b7bf60a8696bb, region_id=RegionOne) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.347080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.349944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.356548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 494/975] 10.222.0.25 () {56 vars in 1249 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/endpoints?service_id=b842a10ce220413aa94b7bf60a8696bb&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:26.362155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.402540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3222-c5af-4e64-891a-af1f3181149d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.427573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.442331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.443131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.443412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.443685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.446864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.455792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:26.460579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.461720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:26.462676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 482/976] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:26.467920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.481631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffeefc-4eb6-4993-9a4c-6a23d4b1145c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.517308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.517913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.518190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.518461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.519049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.520408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.526158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 495/977] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:34:26.527867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.582240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c025f-8554-4c59-b50c-b4289f50dd38 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.622022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.623087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.623369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.623662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.624575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.626405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.628050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.645343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 483/978] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:26.646958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.651995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693489b-a288-4712-bd9c-2b8053927f6b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.672494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.696691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.697435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.697711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.697972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.698619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.700106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.717406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.721630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 496/979] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:26 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:26.771046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc9291-1b4f-4f60-84fb-95e73af5ba5d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.815925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.816579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.816860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.817159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.830043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.831526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:26.847864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.848915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:26.849914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:26.849914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.849914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:26.849914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 484/980] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:26.856077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.961516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a362b-fdcf-4967-8c8e-9e4af051860a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.997894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.998598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.998953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.999297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.000001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.000370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.001834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.018653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 497/981] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.019878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.131743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3a47a-780e-468c-bfce-cd3306d66cc5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.139682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.168311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.169067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.169446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.169791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.177616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.179126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:27.191395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.192543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:27.193765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:27.194521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 485/982] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.201338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.242498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.304930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857d22a-75f4-4c47-a104-7ae4aeb9c4f2 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.340181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.340783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.341120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.341396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.342110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.342411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.343830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.360861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 498/983] 10.222.0.25 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.366312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.466093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf5b6-5d42-4aed-95ff-32b1d3333eba None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.505115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.505643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.505926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.506201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.540377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.578239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.579707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.683830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 486/984] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:27.685650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.777734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.784738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77805a03-da27-42d4-b155-1c2f64804ca5 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.820698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.821744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.822096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.822441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.834948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.836611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:27.854564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.855740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:27.857536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:27.857536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:27.857536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:27.858218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 499/985] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.863870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.930321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.966520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2ef0a-56cd-47c4-acaa-253423962fe8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.999251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.999858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.000135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.000405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.001063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.001363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.002700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.014564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 487/986] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:28.019883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.129644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4555126-cd98-4e1c-be56-bd9a5d226cd3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.163477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.164104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.164380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.164650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.171364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.172797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:28.184517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:28.185508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:28.186859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:28.189047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 500/987] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:28.192464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.294968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b2130-6864-4822-b616-fdee12dc77d2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.320694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.331111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.331718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.331995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.332267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.332966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.333282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.334721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440274e3-9f5f-4bdb-83f8-e3b1582ed14e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.347192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 488/988] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.352824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.364329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.454406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b6db-c693-4757-ad79-8d3832fa88dc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.476010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd188-5da8-4db4-975d-2c0dc5ee4797 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.476310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd188-5da8-4db4-975d-2c0dc5ee4797 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.476611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd188-5da8-4db4-975d-2c0dc5ee4797 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.479134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 489/989] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:28.483103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4942d-24a4-44b6-8c72-1177a6f25586 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.483394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4942d-24a4-44b6-8c72-1177a6f25586 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.483668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4942d-24a4-44b6-8c72-1177a6f25586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.488002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.488605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.488882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.489179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.489931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] RBAC: Adding query filter params (role.id=f8aa2a4fa2864b8f9472faee4c3f0f33, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=4128d9697d5e4f4b81eb3b46ec7b95b0) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.490192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.491661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a64ab4-e9d9-445c-97e6-7fd519d469dc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.498243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 501/990] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/role_assignments?user.id=4128d9697d5e4f4b81eb3b46ec7b95b0&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.504179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8fa4942d-24a4-44b6-8c72-1177a6f25586 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:28.505507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8fa4942d-24a4-44b6-8c72-1177a6f25586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:28.582345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 490/991] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:28.679434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8915f3-221b-4383-8fb4-c9e781a72d21 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.679731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8915f3-221b-4383-8fb4-c9e781a72d21 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.680009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8915f3-221b-4383-8fb4-c9e781a72d21 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.681389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 502/992] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:28.687251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.687538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.687813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.706940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:28.708114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:28.773088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 491/993] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:28.858147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8915f3-221b-4383-8fb4-c9e781a72d21 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.930802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.963391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8915f3-221b-4383-8fb4-c9e781a72d21 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.998134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.998769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.999047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.999320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.999892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.001441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.011178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:34:29.012032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.016162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 503/994] 10.222.0.25 () {56 vars in 1126 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:29.017420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.038627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.066054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.125592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025e6b6-a534-479f-bba1-4610dadbc419 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.151383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bdc65-1a71-44f7-9b42-4c403b1c6471 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.160683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.161334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.161610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.161882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.162506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.162811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.164165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.170064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 492/995] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.175787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.188277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.188832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.189140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.189403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] PATH_INFO: `/v3/services/compute` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.189973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.191361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.194214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.201950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.202974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:29.203828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 504/996] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:29.209362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.286760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a21f-da4c-4517-ac14-0b656f48106f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.299110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.311467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9bb9-ad2c-4658-b5b0-8701c63b04dc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.324894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.325855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.326292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.326725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.327663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.327972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.329452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.339391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 493/997] 10.222.0.25 () {56 vars in 1122 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:29.340208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.345736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.346350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.346633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.346909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.347530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.347833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.349299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.359496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 505/998] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:29.360874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.423825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.457499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35173a-cb2c-488e-9446-ad6aee5c10b2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.462521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6580a-1fcc-47f7-97ee-6a6eb66416d9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.492617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.493249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.493528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.493812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.494458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.495795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.501034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.501653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.501931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.502205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.502836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.503137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.504706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.514971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 506/999] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.515996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.521020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 494/1000] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:29.633738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3569-9e03-48a1-9bc3-74a0a93cd3f8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.662953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.679647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.680340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.680688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.681055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.681813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] RBAC: Adding query filter params (interface=public, service_id=02ef953dcb084f32a0b538055e63b244, region_id=RegionOne) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.682192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.683617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00608b2a-db31-48cf-a281-89e7d984894f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.688824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 507/1001] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/endpoints?service_id=02ef953dcb084f32a0b538055e63b244&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.694758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.731299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.799338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2d2f-69d2-48c2-b52c-16553091d9ce None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.833078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.833768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.834133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.834472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.835141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.836563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bdbf-a3a4-4e4a-81cc-45b54d3179d1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.843567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 495/1002] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:34:30.485888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c75203-eee4-4492-9626-0bf84111e496 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.486413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c75203-eee4-4492-9626-0bf84111e496 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.486413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c75203-eee4-4492-9626-0bf84111e496 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.487930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 508/1003] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:30.492820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.493248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.493415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.512105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:30.513363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:30.582416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 496/1004] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:30.851176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c75203-eee4-4492-9626-0bf84111e496 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.961892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c75203-eee4-4492-9626-0bf84111e496 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.995921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.996546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.996827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.997137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] PATH_INFO: `/v3/services/image` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.997727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.999086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:31.007894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:31.008897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:31.010101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:31.011304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 509/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:31.014843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.042916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.122587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263964-b732-496f-8d3b-b40c3e1ed063 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.158322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.158939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.159220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.159493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.160123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] RBAC: Adding query filter params (name=image) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.160427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.161814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab068e32-2ad4-4006-99d1-c625c7e0e926 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.171430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 497/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.172722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.229584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.280606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab2b7b-e3c3-47d0-ab31-8b57700e2a20 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.315371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.315988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.316266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.316535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.317173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] RBAC: Adding query filter params (type=image) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.317474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.318841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0ad6-6157-48f7-b1a8-3e7f898a5bea None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.324313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 510/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.499693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39c6e8-1984-4c00-ac92-06739d28a060 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.500206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39c6e8-1984-4c00-ac92-06739d28a060 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.500284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39c6e8-1984-4c00-ac92-06739d28a060 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.501748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 498/1008] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:31.506864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09d3f-c306-46e6-903f-a74e28c4abb4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.507267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09d3f-c306-46e6-903f-a74e28c4abb4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.507448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09d3f-c306-46e6-903f-a74e28c4abb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.526126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-bbf09d3f-c306-46e6-903f-a74e28c4abb4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:31.527311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bbf09d3f-c306-46e6-903f-a74e28c4abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:31.594973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 511/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:31.740178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8612f4-922f-4423-aa0b-c9a787e7ca7e None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.740496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8612f4-922f-4423-aa0b-c9a787e7ca7e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.740725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8612f4-922f-4423-aa0b-c9a787e7ca7e None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.742218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 499/1010] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:31.747643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.747943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.748214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.767741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:31.768835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:31.810633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.836609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 512/1011] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:31.866739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8612f4-922f-4423-aa0b-c9a787e7ca7e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.965920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.976854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8612f4-922f-4423-aa0b-c9a787e7ca7e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.012751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.013403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.013684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.013955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.015051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.016501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.018302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 500/1012] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.023952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.099458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.130690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ba938-708a-48d9-8090-0e6cde962d1a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.165348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.165954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.166234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.166752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.167147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.168579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.201187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 513/1013] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:32 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:32.219519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed747fb-420b-40d4-9c1c-a6397213da05 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.260346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.261046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.261335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.261612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] PATH_INFO: `/v3/services/compute` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.262207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.263624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:32.273362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:32.274416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:32.278113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 501/1014] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:32.281117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.314531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.396335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8ace0-537b-4ed7-bb1f-86e6ebf8ec9b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.432931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.433568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.433845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.434115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.435273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.435573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.436933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.442190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 514/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:32.447995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.474982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.552674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d73b2-38fe-4b07-98d7-262d18d0d53d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.587510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.588119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.588395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.588664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.589306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.589605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.591492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113370fa-6f08-4af2-b003-2916b856fa42 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.597724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 502/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.602229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.697472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.707737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9940635-9312-42fe-9e5a-26886a40a963 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.741477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.742094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.742377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.742652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.743328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.744692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a1f0d-bb3f-426e-8675-1d94608c1b61 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.773042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 515/1017] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:33.265755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f62e-d665-436b-8d2c-5fbc67ed242c None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.266100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f62e-d665-436b-8d2c-5fbc67ed242c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.266307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58f62e-d665-436b-8d2c-5fbc67ed242c None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.267790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 503/1018] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:33.272942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2ec5-0dd3-4d7f-8613-12b346cea2ba None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.273257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2ec5-0dd3-4d7f-8613-12b346cea2ba None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.273532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d2ec5-0dd3-4d7f-8613-12b346cea2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.292506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e07d2ec5-0dd3-4d7f-8613-12b346cea2ba None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:33.293708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e07d2ec5-0dd3-4d7f-8613-12b346cea2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:33.373203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 516/1019] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:33 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:33.630776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58f62e-d665-436b-8d2c-5fbc67ed242c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.724198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.738713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58f62e-d665-436b-8d2c-5fbc67ed242c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.774190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.774895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.775123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.775403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.776059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.777459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669aa86-b619-4e90-aea9-296dc5f24da2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.797003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 504/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:33 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:34.163102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937541-8a00-4865-bd52-c06bf68bc998 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.163657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937541-8a00-4865-bd52-c06bf68bc998 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.163657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937541-8a00-4865-bd52-c06bf68bc998 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.165140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 517/1021] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:34.170115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.170409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.170687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.188539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:34.189758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:34.263535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 505/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:34.532263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937541-8a00-4865-bd52-c06bf68bc998 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.605621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.633105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937541-8a00-4865-bd52-c06bf68bc998 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.667353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.667985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.668264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.668536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.669696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.671131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1f5e1-f11d-497c-a4cd-add3d6eccf4e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.672842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 518/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.678324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.756596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.785089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbee3ab-ee90-464f-b84b-77b507fa8e6e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.785799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ec73-92eb-404b-a428-c6bd8ff67d22 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.786089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ec73-92eb-404b-a428-c6bd8ff67d22 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.786364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ec73-92eb-404b-a428-c6bd8ff67d22 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.787700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 519/1024] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:34.792496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.792785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.793107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.811620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:34.812823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:34.824204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.825064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.825510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.825924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.826730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.828285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.853013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 506/1025] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:34 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:34.888567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 520/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:35.169093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.274165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8ad6-71f8-457f-866d-24d9d226c53a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.303214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.311826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.312474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.312758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.313062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.324259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:35.330311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.331275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:35.332127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:35.336406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 507/1027] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:35.337449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.410181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.442095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ceac3-e7a5-4819-b6df-a04248503d74 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.477285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.478115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.478570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.479011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.479812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.480263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.481838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.494712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 521/1028] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.496005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.599010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb55df-93c1-4083-9ce8-13d18cc812ae None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.633752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.634860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.635388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.635878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.651237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.653481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ebbd3015-845f-4c3b-a1ab-efc6dcdf674c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:35.669794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.670750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:35.671666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:35.671666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:35.671666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:35.671666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 508/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:35.678595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.751097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdbe5a-5dae-494b-ba2a-0aaca12769fa None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.751611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdbe5a-5dae-494b-ba2a-0aaca12769fa None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.751611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdbe5a-5dae-494b-ba2a-0aaca12769fa None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.753107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 509/1030] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:35.758021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.758308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.758584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.777258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:35.779010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:35.787878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87a36f-f47f-4599-920a-fb96cf02bb1a None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.816787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.829279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.829888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.830172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.830746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.831081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.831386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.832759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.834239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.846483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 522/1031] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:35.851722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.854521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 510/1032] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:35.953705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a41e9-8d0e-4c3b-8496-0c5ae63ec1c3 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.988139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.988773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.989093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.989373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.997724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.999231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:36.012171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.013237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:36.014218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 523/1033] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:36.020589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.126700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.133426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9401e4-75c5-4a65-ab2c-7b6686c4be91 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.171734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.172342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.172619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.172898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.173630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.173937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.175336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.189804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.192250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 511/1034] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.198068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.227941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.241676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab7ea3-36c4-4ac5-a0f2-27ac7ffbec03 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.278816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.279751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.280036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.280465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] PATH_INFO: `/v3/services/image` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.281060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.282688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.292886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.294857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:36.295994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 524/1035] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:36.300547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.316262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c9474-292a-4188-94e1-0cf375738f3c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.355049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.355665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.355945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.356219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.410576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304f893-e1ff-48c9-93fe-ff56c58c0312 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.427670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.432499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.433973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.447100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.447705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.447996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.448249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.448864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] RBAC: Adding query filter params (name=image) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.449197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.451101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.456332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 525/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:36.461745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.466401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.542690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 512/1037] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:34:36.548268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.573130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0af7e-c4db-4094-a87d-39771ed07172 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.609103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.609716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.610027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.610252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.610874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] RBAC: Adding query filter params (type=image) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.611223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.612584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.617963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 526/1038] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:36.623664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.654763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad23b2-c7d4-4b34-93f7-70a2f39a8952 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.672289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.688393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.695201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.696088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.696368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.696641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.707738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.709197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:36.727091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.728123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:36.729103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:36.729103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:36.729103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:36.729103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 513/1039] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:36.734475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.734891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebb19-ef65-4ecc-b3e6-9ed3253f4838 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.771615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.772222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.772495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.772766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.773480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] RBAC: Adding query filter params (interface=public, service_id=8977d335be734f93ab1cc44463044bc1, region_id=RegionOne) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.773783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.775137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.781022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 527/1040] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/endpoints?service_id=8977d335be734f93ab1cc44463044bc1&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:36.786750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.841673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557c53e-622f-42a2-b726-90fde13a1014 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.876758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.877391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.877672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.877942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.878646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.878946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.880285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b4732-a0c6-4ed6-aba2-29dd024129c3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.891738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 514/1041] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:36.893095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e6c5-bed6-49dc-bac2-896a10436dd5 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.893378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e6c5-bed6-49dc-bac2-896a10436dd5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.893645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e6c5-bed6-49dc-bac2-896a10436dd5 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.894951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 515/1042] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:36.897277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342e6c5-bed6-49dc-bac2-896a10436dd5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.897904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2d5-5d16-462b-b204-f62a094dff11 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.909852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.934023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.934653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.934936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.935210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.935807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.937244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb07c-2f01-4cd5-b56c-f1d99ce9352c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.942618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.943695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 528/1043] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:34:36.944847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.945161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.945440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.965449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:36.967185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:37.016500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342e6c5-bed6-49dc-bac2-896a10436dd5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.050275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 529/1044] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:37.055545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.056194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.056478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.056754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.064419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.065915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:37.079472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.080455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:37.081419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 516/1045] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:37.089494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.193193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266b20d-b1b6-4937-a831-e2cee79a1fff None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.196647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.231406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.232017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.232297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.232572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.233291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.233601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.235047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.251715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 530/1046] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.252901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.336667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.360758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb80356-4bf8-4209-9550-c4c1cdcad28c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.362872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.398713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.399315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.399590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.399865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.400584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] RBAC: Adding query filter params (role.id=f8aa2a4fa2864b8f9472faee4c3f0f33, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.400890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.402443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.409053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 517/1047] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:37.450631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b811c3-cbfe-4e93-9e61-179aa7ec0b74 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.488352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.489327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.489787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.490240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.502962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:37.508220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.509417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:37.510475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:37.512605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 531/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:37.519326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.626578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf4f4-ba07-4d69-ae7c-ae36317e1eb2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.660911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.661544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.661824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.662099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.662755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.663059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.664403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.673032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 518/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.678641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.706605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.784219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560644b-20c9-4f9d-9887-20e4c358f19d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.818789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.819412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.819687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.819957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] PATH_INFO: `/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.833508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] RBAC: Authorizing `identity:get_user(user_id=30fc9055cf734d3bb5cb73bd5cd4e6da)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.835008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.836795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 532/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.843030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.872652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.949105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d704ec-ae81-4381-8c39-5263ee5dcc7a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.984358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.984884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.985187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.985457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.054899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.056398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.092561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.170295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 519/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:34:38.171866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.256900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.276571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf819f2-c1fa-4004-a42c-96a58bd2794b None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.311109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.311741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.312018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.312290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.313444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.314876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020524-bf7e-4666-8ee1-3f2f27f9e549 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.321782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 533/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.323280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.424257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0013f-cf09-474c-a078-2744dd829d05 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.452665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.461923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.462533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.462808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.463079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.463794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=90e3719879834be6b187ed3a08bfa737, user.id=30fc9055cf734d3bb5cb73bd5cd4e6da) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:38.464096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.465610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cab30-1403-4f71-87ac-ea5339e5d7e6 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.477057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 520/1053] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/role_assignments?user.id=30fc9055cf734d3bb5cb73bd5cd4e6da&scope.project.id=90e3719879834be6b187ed3a08bfa737&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.879146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad771f04-5848-4651-930d-7eed0d004e08 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.879698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad771f04-5848-4651-930d-7eed0d004e08 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.879698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad771f04-5848-4651-930d-7eed0d004e08 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.881156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 534/1054] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:38.886083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af74-966d-465b-908b-ca39de6a5335 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.886477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af74-966d-465b-908b-ca39de6a5335 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.886642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4af74-966d-465b-908b-ca39de6a5335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.904651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6bd4af74-966d-465b-908b-ca39de6a5335 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:38.905853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6bd4af74-966d-465b-908b-ca39de6a5335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:38.972853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 521/1055] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:39.239232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad771f04-5848-4651-930d-7eed0d004e08 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.334023 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0362a-7162-4455-a51b-b6a7a4a11d9a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.334409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0362a-7162-4455-a51b-b6a7a4a11d9a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.334778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0362a-7162-4455-a51b-b6a7a4a11d9a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.336998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 522/1056] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:39.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.342125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.342469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.352919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad771f04-5848-4651-930d-7eed0d004e08 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.360658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:39.362002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:39.388512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.389497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.389781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.390054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.390619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.391984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.400708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.401792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:39.402647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 535/1057] 10.222.0.25 () {56 vars in 1105 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:39.407566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.425101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.445597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 523/1058] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:39.514294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e65c2d-33f8-42d6-bb4d-3815096a70e6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.548900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.549899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.550328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.550749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.551536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.551988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.553570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.560154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 536/1059] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:39.565636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.618487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.667468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce5db8-5e65-48dc-809e-aac05905313b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.702668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.703272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.703605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.703881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.704497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] RBAC: Adding query filter params (type=image) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.704798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.706162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.716073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 524/1060] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:39.717273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.720428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.800013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.825115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed1a4-abed-4201-8750-f505f07601f2 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.827927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8188d-a6fc-4e8f-8e0a-d0a577a1eb22 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.845141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.863614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.864181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.864328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.864516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.864796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.865563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.866034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.866644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.867627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.869241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.874946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:39.879843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.880868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:39.881764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 525/1061] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:39.886877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.894148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 537/1062] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:39.996921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111966c-fecc-4f1e-ad4b-6a49f57f1f87 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.039273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.039882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.040159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.040429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.041110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.041440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.042754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.052877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 526/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:40.058480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.165875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705dcff-698c-4893-bd34-1ebc06550155 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.199321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.200486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.200764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.201119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.202960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.214352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.217101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-76cd2b0f-3b74-416b-a37b-7f0c002abe75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:40.232408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.233663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:40.234561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:40.234561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.234561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:40.234561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 538/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:40.239379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.276297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.350469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc9897-1629-4a64-b0f7-aa08f72a2fa7 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.385333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.386120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.386401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.386675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.387322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.388163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.388760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c100b-cd25-44cb-bcad-ccfb9286b7f7 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.389105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c100b-cd25-44cb-bcad-ccfb9286b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.389385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c100b-cd25-44cb-bcad-ccfb9286b7f7 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.389540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.390905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 539/1065] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:40.395982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.396268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.396542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.402057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 527/1066] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.408139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.415335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:40.416510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:40.484978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 540/1067] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:40.506345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.514148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb073f6-bf04-403d-af53-7b4c2a204c4f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.548630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.549287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.549567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.549841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.557101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.558515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:40.570659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.571873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:40.572855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:40.574139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 528/1068] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:40.578630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.681437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8619f-a6e8-4bdb-b66a-0f4971665fcc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.698882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.717707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.718314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.718593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.718866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.719584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.719887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.721351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.738430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 541/1069] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.743965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.757090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.853420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f307-a1c9-4a2d-8ebb-b28a33ac76f5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.860998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01050c04-0919-4ecd-b950-04d2de29e315 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.893324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.893973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.894238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.894514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.897839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.898467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.898769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.899038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.905523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.906954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.910286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:40.915054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.916001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:40.916877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:40.921113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 542/1070] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:40.922560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:40.924714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.925666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:40.926546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:40.926546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.926546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:40.926546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 529/1071] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:40.931819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.941606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.963648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.040488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394003-c6a6-4eab-8a08-f3757d0665ca None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.041228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7498fa4-728e-4218-b36f-16d85ceb083f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.080068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.080680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.080987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.081267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.081824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.081939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.082218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.082427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.082704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.082975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.086394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.086513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.086513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.086513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.094490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 543/1072] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.102419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 530/1073] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.103821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.103963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.177534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.204252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.211897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340b0a5-7228-49be-94c3-a35f3025e99e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.213416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541d59-1e5c-4c9f-96fc-5496ae62b1b8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.249045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661852d1-b499-4cac-943a-659054852812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.250007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661852d1-b499-4cac-943a-659054852812 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.250488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.250752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661852d1-b499-4cac-943a-659054852812 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.251122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.251407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.251628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661852d1-b499-4cac-943a-659054852812 None admin] PATH_INFO: `/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.251739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.253046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661852d1-b499-4cac-943a-659054852812 None admin] RBAC: Authorizing `identity:get_user(user_id=30fc9055cf734d3bb5cb73bd5cd4e6da)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.254668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661852d1-b499-4cac-943a-659054852812 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.256493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 544/1074] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.258512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.260166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.262178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661852d1-b499-4cac-943a-659054852812 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:41.274371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:41.275365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:41.276269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 531/1075] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:41.281685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.371413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661852d1-b499-4cac-943a-659054852812 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.391089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fbc7f-1958-42e3-ae2b-37e3ce640b8c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.408653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.409308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.409586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.409856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.410956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.412389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.414240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 545/1076] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.420074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.420310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.430602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.431467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.431467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.431722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.432405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.432707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.434281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.447203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 532/1077] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:41.453382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.453686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.539755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1770d6a-8e47-44d4-89ef-c9dfbc960e21 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.558314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d3839-9da3-4654-879a-06a4c7cb7ec5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.575825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.576490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.576828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.577151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737/users/30fc9055cf734d3bb5cb73bd5cd4e6da/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.579093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] RBAC: Authorizing `identity:create_grant(project_id=90e3719879834be6b187ed3a08bfa737, user_id=30fc9055cf734d3bb5cb73bd5cd4e6da, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.580530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862e187-f6c9-4efe-a024-8b9db0f79262 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.597121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.604503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 546/1078] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:41 2022] PUT /identity/v3/projects/90e3719879834be6b187ed3a08bfa737/users/30fc9055cf734d3bb5cb73bd5cd4e6da/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:41.604725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.604725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.604725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/f8aa2a4fa2864b8f9472faee4c3f0f33` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.604725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=ad5260213b9f4ec49f3ce2ec48945d7d, role_id=f8aa2a4fa2864b8f9472faee4c3f0f33)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.604725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6abb72-872c-4ce7-88d1-3a4e0e369cc9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.618684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 533/1079] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:41 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/ad5260213b9f4ec49f3ce2ec48945d7d/roles/f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:41.847802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940268-1946-4daa-9f74-bc378e3c7dc4 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.847988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940268-1946-4daa-9f74-bc378e3c7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.848272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940268-1946-4daa-9f74-bc378e3c7dc4 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.849776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 547/1080] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:41.854754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.855111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.855463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.873911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:41.875094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:41.919669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.946940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 534/1081] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:42.216525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b940268-1946-4daa-9f74-bc378e3c7dc4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.311596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.323907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b940268-1946-4daa-9f74-bc378e3c7dc4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.357321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.357920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.358198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.358477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.425078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.426529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.514545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.529593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 548/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:42.531305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.632982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df07711-1761-48be-abf0-43891beac1f6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.666571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.667211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.667489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.667761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.679747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.681265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-49f73b8b-4be5-4617-9d39-8bc2267d7892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:42.698328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:42.699325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:42.705459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:42.705459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:42.705459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:42.705459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 535/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:42.707511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.810752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a09572-3ef4-4221-8182-f680828f2fb9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.844392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.845013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.845295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.845567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.846192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:42.846493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.847853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff021db-649e-4ecb-9786-fbcd6a833d6b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.859721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 549/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:43.510879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12532a0-9c51-40a2-8cbc-6872e6b86de3 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.511410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12532a0-9c51-40a2-8cbc-6872e6b86de3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.511410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12532a0-9c51-40a2-8cbc-6872e6b86de3 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.513420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 536/1085] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:43.518663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.519145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.519272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.537117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:43.538283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:43.551080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4023-9ae8-4ee2-bd21-c3923311837a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.551358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4023-9ae8-4ee2-bd21-c3923311837a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.551624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4023-9ae8-4ee2-bd21-c3923311837a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.552921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 537/1086] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:43.557696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.557973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.558243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.577564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:43.578757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:43.586189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:43.601204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:43.607536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 550/1087] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:43.644883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 538/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:43.872477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:43.913238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:43.970202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a8375-8878-4e94-bf85-adae67f106db None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.005469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.006098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.006380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.006655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.010417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa9c92-5c79-42f0-84ec-c4368de8ccb0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.017533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:44.022353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.023273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:44.024210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 551/1089] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:44.029641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.046971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.047600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.047878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.048147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.061437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:44.066172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.067131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:44.068025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 539/1090] 10.222.0.25 () {56 vars in 1120 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:44.073762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.076411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.095147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.136111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27935cf3-d4fc-40c4-a5af-86f28fa21faf None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.168077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.169231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.169511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.169779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.170429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.170728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.172094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.178265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589efa41-4c82-4b67-b96b-111c9d0ece35 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.180359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 552/1091] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.186660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.212086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.212679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.212970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.213257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.213905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.214202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.215576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.223186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 540/1092] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.228774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.283637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89443-2339-4331-bb8a-b8ba7138a1bb None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.285997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.309667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.318177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.318807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.319085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.319356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] PATH_INFO: `/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.320503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] RBAC: Authorizing `identity:get_user(user_id=30fc9055cf734d3bb5cb73bd5cd4e6da)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.322230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.323785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 553/1093] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/users/30fc9055cf734d3bb5cb73bd5cd4e6da => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.329363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7209-dd69-4fd1-98d1-dc729bdc359c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.329603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.364173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.364806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.365212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.365496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.376497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.377966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:44.391922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.393151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:44.394284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:44.394284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:44.394284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:44.394284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 541/1094] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:44.398856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.429550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ed9b-3ceb-4287-ba30-08079a096f75 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.462937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.463537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.463813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.464078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.499891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9643-adbf-4fa1-b4a1-14a3cba3dfa0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.504381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.529124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.532294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.533768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.534788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.535395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.535674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.535948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.536616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.536918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.538290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.550733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 542/1095] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.556362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.632906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 554/1096] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:34:44.636770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.656659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8852a4-7664-4b21-9c75-fe7fa6d58254 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.690324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.690945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.691222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.691501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.698762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.700681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.704742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:44.712972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.714109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:34:44.715092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 543/1097] 10.222.0.25 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:44.720323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.739749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fdafb-0f8d-49d8-8dad-6b562a2890c0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.771575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.774371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.774998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.775289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.775561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.776660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.778686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.780303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 555/1098] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.786233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.818517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e6a2e-f250-4dca-9ac5-98ea3deff2d8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.856473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.857116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.857398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.857673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.858395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.858700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.860111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.876875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 544/1099] 10.222.0.25 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.882522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.888614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4a416-b356-4c66-824a-70acdb3c3ac4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.922331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.922863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.923148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.923420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.924197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=90e3719879834be6b187ed3a08bfa737, user.id=30fc9055cf734d3bb5cb73bd5cd4e6da) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.924506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.925988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.931868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 556/1100] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/role_assignments?user.id=30fc9055cf734d3bb5cb73bd5cd4e6da&scope.project.id=90e3719879834be6b187ed3a08bfa737&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:44.984428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d70f-51bd-499b-8fd1-b9b1555c03e3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.996356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.022589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.023214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.023494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.023766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.097468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:45.098969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:45.196378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 545/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:45.202777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.206548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.298659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa4ef-921c-4c5b-a9d2-e6dba6435189 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.332410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.333060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.333336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.333600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.343781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:45.345290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:45.358910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:45.360046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:45.361414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:45.361414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:45.361414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:45.361414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 557/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:45.365653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.377556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.466579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc080-245e-413a-93b4-37ce38345ccc None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.500100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.500692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.500986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.501261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.501887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:45.502183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:45.503527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:45.515619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 546/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:45.521284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.578043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.621209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977134c-fab4-47ce-ae6a-439a4b349f98 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.656946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.657704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.658192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.658665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.667108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:45.669109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:45.680226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:45.681323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:34:45.682241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 558/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:45.687801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.762425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.794951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4e75a-454c-45da-b84c-88aeb277e387 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.833548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:45.834162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.834447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.834724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.835464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:45.835773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:45.837210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71241-28c3-4fe5-bb59-166e8436bdb8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:45.849858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 547/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:45.856723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.965017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc76fe-4469-4e74-a85c-9f8e41fdc8bd None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:45.984135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.003810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.004422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.004706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.005007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.005766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] RBAC: Adding query filter params (role.id=f8aa2a4fa2864b8f9472faee4c3f0f33, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=ad5260213b9f4ec49f3ce2ec48945d7d) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.006075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.007511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c118500-dd2b-4f27-8ca6-df017000201b None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.013771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 559/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/role_assignments?user.id=ad5260213b9f4ec49f3ce2ec48945d7d&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=f8aa2a4fa2864b8f9472faee4c3f0f33 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.983320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01756237-d775-4baf-a1ed-5520dd70fff4 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.983851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01756237-d775-4baf-a1ed-5520dd70fff4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.983851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01756237-d775-4baf-a1ed-5520dd70fff4 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.985239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 548/1107] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:46.990287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.990665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.990839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.008588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:47.009805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:47.075831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 560/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:47.346245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01756237-d775-4baf-a1ed-5520dd70fff4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.444438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01756237-d775-4baf-a1ed-5520dd70fff4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.472377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.478532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.479161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.479445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.479722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.481480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.482938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1579c6-5e51-4264-af84-3d9188d6f378 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.484527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 549/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.490296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.586181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0869abe-06f0-4875-89ec-ad5522671d87 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.596924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.621192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.621880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.622163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.622436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.623334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.624767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb67c73-fb3d-46e6-a6dd-aa787202caed None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.645200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 561/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:47 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:49.578447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3558c8-6397-4b98-9068-75aa52507180 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.578974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3558c8-6397-4b98-9068-75aa52507180 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.578974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3558c8-6397-4b98-9068-75aa52507180 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.580326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 550/1111] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:49.585499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.585904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.586057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.603790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:49.605000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:49.669936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 562/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:49.943613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3558c8-6397-4b98-9068-75aa52507180 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.041361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3558c8-6397-4b98-9068-75aa52507180 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.073697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.074325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.075091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.075361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] PATH_INFO: `/v3/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.076732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] RBAC: Authorizing `identity:get_role(role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.078109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.079415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 551/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.085107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.174702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.181185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc8ae9-525a-49f6-afa9-5536d7a4b02a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.214239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.214794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.215068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.215868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] PATH_INFO: `/v3/users/72c022bde7e14328a0827564cd94cdea` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.228225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] RBAC: Authorizing `identity:get_user(user_id=72c022bde7e14328a0827564cd94cdea)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.229680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.231244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 563/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/users/72c022bde7e14328a0827564cd94cdea => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.236943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.300365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.335975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786eb1c4-995d-44ec-943d-b391c898955f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.368097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.368692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.368997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.369277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.433779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.435721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.495677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.531199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 552/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:50.537414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.634506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97148f-a7d0-4883-be0f-ded8bbb5292c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.667203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.667818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.668099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.668948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.670085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.671515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3b7a5-f19f-43ec-bac1-46fd4ab02e47 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.673121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 564/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.679106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.776314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8643-c69c-46bd-b6ed-4a372e4751f5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.809586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.810180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.810462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.810732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.811447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=90e3719879834be6b187ed3a08bfa737, user.id=72c022bde7e14328a0827564cd94cdea) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.811754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.813178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92299879-a7d2-4fba-b20a-12acedac2568 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.818408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 553/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/role_assignments?user.id=72c022bde7e14328a0827564cd94cdea&scope.project.id=90e3719879834be6b187ed3a08bfa737&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:52.695060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7832-6b10-46ed-8b4a-9c35d4b27ff5 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.695906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7832-6b10-46ed-8b4a-9c35d4b27ff5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.695906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7832-6b10-46ed-8b4a-9c35d4b27ff5 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.697217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 565/1118] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:52.702930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.703380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.703494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.721161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:52.722428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:52.744857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.786633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 554/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:53.061490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf7832-6b10-46ed-8b4a-9c35d4b27ff5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.085778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.169625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf7832-6b10-46ed-8b4a-9c35d4b27ff5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.202699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.203319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.203602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.203877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] PATH_INFO: `/v3/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.205309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] RBAC: Authorizing `identity:get_role(role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.206709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.208024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 566/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:53.213839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.309673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b33bc2-03d4-4190-8a8d-c9aad1c9c8da None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.342622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.343183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.343466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.343744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] PATH_INFO: `/v3/users/72c022bde7e14328a0827564cd94cdea` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.344844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] RBAC: Authorizing `identity:get_user(user_id=72c022bde7e14328a0827564cd94cdea)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.346295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.347846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 555/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/users/72c022bde7e14328a0827564cd94cdea => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:53.353483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.440612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.450646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1872-170b-487e-8d7a-fe41c6878309 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.484416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.485050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.485332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.485603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.486687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.488107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d53f-8441-4a03-89b5-3b7ffbb21239 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.489732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 567/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:53.495616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.523088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.594118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a319-08e9-4095-a93b-7096ad731f7f None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.627076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.627723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.628018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.628278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737/users/72c022bde7e14328a0827564cd94cdea/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.630548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] RBAC: Authorizing `identity:create_grant(project_id=90e3719879834be6b187ed3a08bfa737, user_id=72c022bde7e14328a0827564cd94cdea, role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.631967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e333e-a972-4675-aa13-b8f94426d56a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.645238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 556/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:53 2022] PUT /identity/v3/projects/90e3719879834be6b187ed3a08bfa737/users/72c022bde7e14328a0827564cd94cdea/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:55.535078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e86f5d-925b-4d57-aed8-ed4242c07e96 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.535630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e86f5d-925b-4d57-aed8-ed4242c07e96 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.535630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e86f5d-925b-4d57-aed8-ed4242c07e96 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.536933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 568/1124] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:55.542071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.542494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.542657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.565592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:55.566773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:55.633050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 557/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:55.908505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e86f5d-925b-4d57-aed8-ed4242c07e96 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.008870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e86f5d-925b-4d57-aed8-ed4242c07e96 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.042029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.042646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.042921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.043719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] PATH_INFO: `/v3/roles/31b8ad3a252e4d668cac18d1070677b2` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.045108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] RBAC: Authorizing `identity:get_role(role_id=31b8ad3a252e4d668cac18d1070677b2)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.046464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.047753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 569/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/roles/31b8ad3a252e4d668cac18d1070677b2 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.053654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.072492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.155903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e9f8b-d9b0-4d1a-acd3-a12dc78a352c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.188912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.189564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.189841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.190112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] PATH_INFO: `/v3/users/72c022bde7e14328a0827564cd94cdea` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.191212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=72c022bde7e14328a0827564cd94cdea)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.192635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.194222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 558/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/users/72c022bde7e14328a0827564cd94cdea => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.200393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.226481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.297509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc104f-5770-42ec-9ed2-423e672a30ff None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.329945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.330540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.330823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.331096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.395412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.396849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.422020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.493398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 570/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:56.499715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.586415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.602353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4c50-59db-4fdc-becf-e43eda20b5c0 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.638534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.639168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.639460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.639732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] PATH_INFO: `/v3/projects/90e3719879834be6b187ed3a08bfa737` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.640841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] RBAC: Authorizing `identity:get_project(project_id=90e3719879834be6b187ed3a08bfa737)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.642298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde791da-e8b3-43f0-8bdd-608353270162 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.643936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 559/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/projects/90e3719879834be6b187ed3a08bfa737 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.650322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.750745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264d4e4-b826-42d9-b064-abd720b20997 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.775648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.785363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.785967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.786244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.786515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.787288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] RBAC: Adding query filter params (role.id=31b8ad3a252e4d668cac18d1070677b2, scope.project.id=90e3719879834be6b187ed3a08bfa737, user.id=72c022bde7e14328a0827564cd94cdea) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:56.787591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.789035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5075f-e2af-4691-9890-29e0c3a5624f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.795124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 571/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/role_assignments?user.id=72c022bde7e14328a0827564cd94cdea&scope.project.id=90e3719879834be6b187ed3a08bfa737&role.id=31b8ad3a252e4d668cac18d1070677b2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.691931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd903-05a1-472c-b9ff-b15f653d03e5 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.692500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd903-05a1-472c-b9ff-b15f653d03e5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.692500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd903-05a1-472c-b9ff-b15f653d03e5 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.693850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 560/1131] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:58.699852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.700257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.700443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.719590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:58.720773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:58.785424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 572/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:59.051127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd903-05a1-472c-b9ff-b15f653d03e5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.151066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd903-05a1-472c-b9ff-b15f653d03e5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.185057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.185669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.185945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.186211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.187293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.188696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b33b28-0543-4bba-bbe3-d8e64a48b6cc None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.190260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 561/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:59.196286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.292045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafbe5c-a963-49b3-ae63-221a52e6b608 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.312287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.326861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.327459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.327766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.328034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.328666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.330122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c206aaa-eebf-4547-8df3-59bff0654857 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.355213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 573/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:59 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:01.371966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293912b-462b-4404-88ef-2843e66e2d2d None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.372469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293912b-462b-4404-88ef-2843e66e2d2d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.372469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293912b-462b-4404-88ef-2843e66e2d2d None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.373793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 562/1135] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:01.382065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.382527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.382632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.403264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:01.404463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:01.469565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 574/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:01.741989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293912b-462b-4404-88ef-2843e66e2d2d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.769985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.845675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293912b-462b-4404-88ef-2843e66e2d2d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.877887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.878493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.878770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.879040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.880117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.881607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d22d56-eacd-4c13-9449-740180506c12 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.883411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 563/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.889501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.984429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62f7f3-a389-4b93-9ff6-fe183dc0ee9c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.986795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.020357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.020896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.021206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.021481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.022131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.023556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1403bdf-ed61-4749-ab85-95bfcfc86bfe None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.044218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 575/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:01 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:03.997123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e962a-ddb9-4bbc-8267-1eb2f93a786a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:03.997697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e962a-ddb9-4bbc-8267-1eb2f93a786a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:03.997697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e962a-ddb9-4bbc-8267-1eb2f93a786a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:03.999004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 564/1139] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:04.004672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.005080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.005259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.023519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:04.024689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:04.088356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 576/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:04.353810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e962a-ddb9-4bbc-8267-1eb2f93a786a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.443790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.454905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e962a-ddb9-4bbc-8267-1eb2f93a786a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.487774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.488394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.488676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.488948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.499414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:04.504196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:04.505375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:04.506622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 565/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:04.511348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.596514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.610034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c69f-5b71-4cf6-b35f-2e6230fabbf9 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.642332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.642932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.643212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.643483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.644131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:04.644439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:04.645955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:04.654194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 577/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:04.659887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.758181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9429ed-e889-417f-8e5b-91d931d04e33 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.791320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.791932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.792211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.792483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] PATH_INFO: `/v3/users/289533d0926b45f9abe1023b43fa2b4d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.805732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] RBAC: Authorizing `identity:get_user(user_id=289533d0926b45f9abe1023b43fa2b4d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:04.807138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:04.808746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 566/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/users/289533d0926b45f9abe1023b43fa2b4d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:04.814649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.911951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ce06-bc32-4fb0-b2d6-6373eaf3aefc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.943508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.944099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.944380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.944652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.963729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.011024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.013027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.112435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 578/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:05.118454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.129189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.217370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d13be7-4f2b-415f-9606-e20f73b485b1 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.250422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.251041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.251321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.251596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] PATH_INFO: `/v3/projects/e65f504836ba402cb9042524193be229` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.252700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] RBAC: Authorizing `identity:get_project(project_id=e65f504836ba402cb9042524193be229)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.254150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6c22b-952a-4b3d-a2bb-e21ce9bea80e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.255754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 567/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/projects/e65f504836ba402cb9042524193be229 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:05.262528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.309991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.359269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15726a9-fc3a-45d7-a6e4-a1f5d440be2c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.392853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.393476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.393755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.394029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.394744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=e65f504836ba402cb9042524193be229, user.id=289533d0926b45f9abe1023b43fa2b4d) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:05.395052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.396467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5afff-887e-400c-bc53-c359a579afc3 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.401646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 579/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/role_assignments?user.id=289533d0926b45f9abe1023b43fa2b4d&scope.project.id=e65f504836ba402cb9042524193be229&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:07.274715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366555-cced-49e9-b235-4830bc1932dd None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.275227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366555-cced-49e9-b235-4830bc1932dd None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.275314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9366555-cced-49e9-b235-4830bc1932dd None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.276581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 568/1147] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:07.282226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.282664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.282759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.300500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:07.301725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:07.368931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 580/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:07.635607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9366555-cced-49e9-b235-4830bc1932dd None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.711422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.734366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9366555-cced-49e9-b235-4830bc1932dd None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.767493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.768108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.768388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.768659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.778447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:07.783168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:07.784291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:07.785572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 569/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:07.790228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.835385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.887995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b647ac-c610-4058-999d-800e6d04b1be None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.920593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.921232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.921514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.921785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.922430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:07.922730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.924086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.931835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 581/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:07.938386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.034012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909abecf-369c-4616-bc4c-3f5cf2f14432 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.067152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.067763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.068043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.068319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] PATH_INFO: `/v3/users/289533d0926b45f9abe1023b43fa2b4d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.069698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=289533d0926b45f9abe1023b43fa2b4d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.071401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.073049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 570/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/users/289533d0926b45f9abe1023b43fa2b4d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:08.078874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.176821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8eb4-244f-4390-87e4-72ffe88cd9dc None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.183989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.210668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.211288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.211570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.211844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] PATH_INFO: `/v3/projects/e65f504836ba402cb9042524193be229` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.212947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] RBAC: Authorizing `identity:get_project(project_id=e65f504836ba402cb9042524193be229)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.214452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca69642-8811-42b7-a524-043e51311d8f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.216034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 582/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/projects/e65f504836ba402cb9042524193be229 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:08.222104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.225838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.319055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cbf0-56f0-4526-8744-710552dbe4c4 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.352257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.352901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.353205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.353482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] PATH_INFO: `/v3/projects/e65f504836ba402cb9042524193be229/users/289533d0926b45f9abe1023b43fa2b4d/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.355968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e65f504836ba402cb9042524193be229, user_id=289533d0926b45f9abe1023b43fa2b4d, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.357424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1dd36-40b9-4e52-a131-bf9abc1915c0 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.369470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 571/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:08 2022] PUT /identity/v3/projects/e65f504836ba402cb9042524193be229/users/289533d0926b45f9abe1023b43fa2b4d/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:10.259554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b346-704e-45da-96fc-c524fc35a6f9 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.260109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b346-704e-45da-96fc-c524fc35a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.260109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b346-704e-45da-96fc-c524fc35a6f9 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.262007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 583/1154] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:10.267172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.267581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.267720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.286702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:10.287865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:10.344219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.352779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 572/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:10.617685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b346-704e-45da-96fc-c524fc35a6f9 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.716462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b346-704e-45da-96fc-c524fc35a6f9 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.748558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.749194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.749475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.749748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.760029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:10.764808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:10.765918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:35:10.767131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 584/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:10.771626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.866243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ed997-5607-4b42-a9be-d5488726970a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.898436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.898978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.899255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.899522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.900169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.900465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.902444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.910133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 573/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:10.915858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.950097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.014830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a58a-b822-4baf-8b1e-616da6698fa8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.047687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.048303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.048584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.048859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] PATH_INFO: `/v3/users/289533d0926b45f9abe1023b43fa2b4d` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.049995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] RBAC: Authorizing `identity:get_user(user_id=289533d0926b45f9abe1023b43fa2b4d)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.051435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.053001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 585/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/users/289533d0926b45f9abe1023b43fa2b4d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.058845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.117152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.160677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b421a0-1f65-46d4-b687-606b71bd254a None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.192550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.193186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.193470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.193745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.259513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.260930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.313694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.356463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 574/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:11.362366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.460660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c3395-a935-4afd-8297-bf260d802c21 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.462178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.494403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.495020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.495303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.496119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] PATH_INFO: `/v3/projects/e65f504836ba402cb9042524193be229` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.497246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] RBAC: Authorizing `identity:get_project(project_id=e65f504836ba402cb9042524193be229)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.498677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abfd9c-f7d0-4f9f-8bbb-62126d6b0ffb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.500272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 586/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/projects/e65f504836ba402cb9042524193be229 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.506308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.602204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb27025-9048-43ba-af75-a423c99eeff9 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.637131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.637721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.637999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.638269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.638981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=e65f504836ba402cb9042524193be229, user.id=289533d0926b45f9abe1023b43fa2b4d) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:11.639285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.640684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c778-8b11-4481-93fd-e0af5598426c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.646273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 575/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/role_assignments?user.id=289533d0926b45f9abe1023b43fa2b4d&scope.project.id=e65f504836ba402cb9042524193be229&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:13.548856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4117d-ad6d-4292-9ffe-c9ffde112363 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.549410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4117d-ad6d-4292-9ffe-c9ffde112363 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.549410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4117d-ad6d-4292-9ffe-c9ffde112363 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.550751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 587/1162] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:13.555939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.556372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.556490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.574531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:13.575690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:13.589779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.641126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 576/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:13.910152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4117d-ad6d-4292-9ffe-c9ffde112363 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.007593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4117d-ad6d-4292-9ffe-c9ffde112363 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.040510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.041125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.041406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.041679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.079581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.108834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.111815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.204111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 588/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:14.209982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.304293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a482d9-27bc-4e13-a51f-ef261b5d656b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.338375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.339006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.339286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.339554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.346001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.347411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-61fe2f67-fa2b-42e1-abbc-e35eef1f8456 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:14.358326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:14.359519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:14.360593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 577/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:14.365290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.431857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.462685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977a6a9-5d65-43b9-bde0-228f62e2dd81 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.496560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.497182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.497462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.497734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.498362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.498668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.500023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe424c-164f-4d38-b804-1719541addee None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.506975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 589/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.378142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5914d0-29dc-4115-a9c0-f307c4ee3e57 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.378640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5914d0-29dc-4115-a9c0-f307c4ee3e57 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.378640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5914d0-29dc-4115-a9c0-f307c4ee3e57 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.379941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 578/1167] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:16.385032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cd40-3e61-439b-99bb-50073a0d78fb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.385401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cd40-3e61-439b-99bb-50073a0d78fb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.385588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3cd40-3e61-439b-99bb-50073a0d78fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.403335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b5d3cd40-3e61-439b-99bb-50073a0d78fb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:16.404511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b5d3cd40-3e61-439b-99bb-50073a0d78fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:16.472835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 590/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:16.741419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5914d0-29dc-4115-a9c0-f307c4ee3e57 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.754175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.848950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5914d0-29dc-4115-a9c0-f307c4ee3e57 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.882270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.882872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.883152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.883425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.884092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.885474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61341883-a588-41c1-9c0b-3ebabb2dffb7 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.902235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 579/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Wed Aug 31 09:35:16 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:18.820012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24c415-ae60-464d-ba8c-f28e268f99de None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.820796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24c415-ae60-464d-ba8c-f28e268f99de None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.820796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24c415-ae60-464d-ba8c-f28e268f99de None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.822176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 591/1170] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:18.827925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.828334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.828481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.846228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:18.847395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:18.907673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:18.912523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 580/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:19.179957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24c415-ae60-464d-ba8c-f28e268f99de None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.242347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.280285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24c415-ae60-464d-ba8c-f28e268f99de None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.313877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.314496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.314776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.315049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] PATH_INFO: `/v3/domains/37b1cc72ecf940acb6f99acfcf30f265` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.325762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=37b1cc72ecf940acb6f99acfcf30f265)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.327203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f9def-f491-42df-8192-031f721574dd None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.328759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 592/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/domains/37b1cc72ecf940acb6f99acfcf30f265 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:19.334532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.429697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43da4e-5e26-41e8-95b5-0c536de5a178 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.462529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.463125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.463406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.463675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.464333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.465760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d53c5-78a0-4ba1-81f1-32115d926a32 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.490993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 581/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:19 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:21.405281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ad1-bd8e-4d2c-aa03-675d921484d1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.405808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ad1-bd8e-4d2c-aa03-675d921484d1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.405808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ad1-bd8e-4d2c-aa03-675d921484d1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.407050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 593/1174] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:21.412324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.412755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.412860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.430688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:21.431867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:21.478834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.498660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 582/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:21.770638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0ad1-bd8e-4d2c-aa03-675d921484d1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.869039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0ad1-bd8e-4d2c-aa03-675d921484d1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.891949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.902755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.903379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.903663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.903939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] PATH_INFO: `/v3/domains/37b1cc72ecf940acb6f99acfcf30f265` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.905079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=37b1cc72ecf940acb6f99acfcf30f265)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.906519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496371d8-7a0c-4178-8a4b-2faba2086501 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.908052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 594/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains/37b1cc72ecf940acb6f99acfcf30f265 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.913716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.010326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b5b1-d43e-40ac-baf1-16f521cb2238 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.044083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.044618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.044892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.045190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.045830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.047234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10ad39-fae5-4c75-9180-1610d283a1fb None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.068489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 583/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:21 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:23.993188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb7f4-5d24-4a43-b56e-f232743b89cb None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.993828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb7f4-5d24-4a43-b56e-f232743b89cb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.993828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb7f4-5d24-4a43-b56e-f232743b89cb None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.995054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 595/1178] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:24.000702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.001116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.001286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.019691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:24.020877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:24.049944 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.088462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 584/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:24.366313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb7f4-5d24-4a43-b56e-f232743b89cb None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.465820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb7f4-5d24-4a43-b56e-f232743b89cb None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.498988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.499614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.499897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.500173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.510665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:24.515960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:24.517161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:35:24.518477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 596/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:24.523081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.619874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b606e5-cf33-4d9f-950f-5abe27790afc None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.652756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.653375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.653657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.653931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.654581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:24.654883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.656222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.664024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 585/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.670308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.721257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.766925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8386a7-33e0-40dc-85ea-e9154105b2ca None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.800512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.801164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.801449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.801722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] PATH_INFO: `/v3/users/163ac0ed3e864fe79ac8b936094a0d03` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.815307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] RBAC: Authorizing `identity:get_user(user_id=163ac0ed3e864fe79ac8b936094a0d03)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.816778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.818370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 597/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/users/163ac0ed3e864fe79ac8b936094a0d03 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.824531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.854670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.921747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbad52-8d1d-43f9-93c7-455bcf170e4e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.953501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.954094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.954376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.954652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.018547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.019971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.048243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.120211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 586/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:25.126285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.231974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf0fd-74cd-4f2a-9a62-d04889d4eacb None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.242336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.265974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.266596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.266875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.267151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] PATH_INFO: `/v3/projects/64c93243156f4ea7902d4e2bb9d5484f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.268246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] RBAC: Authorizing `identity:get_project(project_id=64c93243156f4ea7902d4e2bb9d5484f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.269712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f58ef-d639-487a-ad7d-78fbf92f8940 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.271319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 598/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/projects/64c93243156f4ea7902d4e2bb9d5484f => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:25.278043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:25.372858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c606-98d2-4c05-b805-e19cc133a455 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:25.400990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.407385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:25.407975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.408252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.408518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.409263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=64c93243156f4ea7902d4e2bb9d5484f, user.id=163ac0ed3e864fe79ac8b936094a0d03) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:25.409570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:25.410971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521291-a188-4268-8a93-dc8b3d57a2c2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:25.416189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 587/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:25 2022] GET /identity/v3/role_assignments?user.id=163ac0ed3e864fe79ac8b936094a0d03&scope.project.id=64c93243156f4ea7902d4e2bb9d5484f&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:27.268979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae25e-266d-4108-a5d6-23d4920a6441 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.269501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae25e-266d-4108-a5d6-23d4920a6441 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.269615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae25e-266d-4108-a5d6-23d4920a6441 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.270900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 599/1186] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:27.276156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.276563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.276743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.294432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:27.295615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:27.362136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 588/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:27.628721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae25e-266d-4108-a5d6-23d4920a6441 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.729506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae25e-266d-4108-a5d6-23d4920a6441 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.770549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.771181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.771456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.771721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.781756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:27.786449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:27.787546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:35:27.788812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 600/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:27.793563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.890918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e8a7e-9c7b-4e27-aa30-a0f7aaf39f45 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.912857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.923339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.923934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.924211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.924479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.925161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.925455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.926790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.935059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 589/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:27.941005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.980124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.038581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368de5f-a7f6-4f11-8c97-20a480e11127 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.071559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.072190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.072471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.072745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] PATH_INFO: `/v3/users/163ac0ed3e864fe79ac8b936094a0d03` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.074490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] RBAC: Authorizing `identity:get_user(user_id=163ac0ed3e864fe79ac8b936094a0d03)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.075935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.077539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 601/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/users/163ac0ed3e864fe79ac8b936094a0d03 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.083243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.180113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81d23d-086d-40a3-8bde-f29bc40bfab2 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.212593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.213243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.213526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.213803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] PATH_INFO: `/v3/projects/64c93243156f4ea7902d4e2bb9d5484f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.214915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] RBAC: Authorizing `identity:get_project(project_id=64c93243156f4ea7902d4e2bb9d5484f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.216333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b22d6f-173c-4962-a07c-8184a70758ff None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.217988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 590/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/projects/64c93243156f4ea7902d4e2bb9d5484f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.223983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.322342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3745bb6-9ef3-4a07-a62d-b2b8fc7e2f39 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.329496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.356716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.357399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.357683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.357958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] PATH_INFO: `/v3/projects/64c93243156f4ea7902d4e2bb9d5484f/users/163ac0ed3e864fe79ac8b936094a0d03/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.359830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=64c93243156f4ea7902d4e2bb9d5484f, user_id=163ac0ed3e864fe79ac8b936094a0d03, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.361290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da923222-bdc4-4bff-a6b1-42d9bcc9ddc1 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.374319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 602/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:28 2022] PUT /identity/v3/projects/64c93243156f4ea7902d4e2bb9d5484f/users/163ac0ed3e864fe79ac8b936094a0d03/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:30.236828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183c84-71f5-458c-8eac-404001b134d8 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.237373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183c84-71f5-458c-8eac-404001b134d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.237475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e183c84-71f5-458c-8eac-404001b134d8 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.238721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 591/1193] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:30.243788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.244196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.244347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.263861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:30.265074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:30.330064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 603/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:30.594494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e183c84-71f5-458c-8eac-404001b134d8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.650027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.695405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e183c84-71f5-458c-8eac-404001b134d8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.726993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.727596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.727873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.728143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.739616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:30.744392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.746070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:35:30.747122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 592/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:30.752157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.843429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.849718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff281dc-4a92-40f2-be31-842a8f2ce9c2 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.882814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.883410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.883690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.883963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.884611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:30.884911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.886299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.894015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 604/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:30.899843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.997722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d925c-a0fb-402d-8352-f9423cd938c3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.011193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.031259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.031870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.032148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.032418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] PATH_INFO: `/v3/users/163ac0ed3e864fe79ac8b936094a0d03` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.033531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] RBAC: Authorizing `identity:get_user(user_id=163ac0ed3e864fe79ac8b936094a0d03)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.034937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.036483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 593/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/users/163ac0ed3e864fe79ac8b936094a0d03 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.042954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.141295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49ea4-e6cc-48a6-b611-2ede94ed6105 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.173950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.174556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.174839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.175113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.201895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.242455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.243911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.339621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 605/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:31.345596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.440182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217d2a-5b25-49a3-9963-fe25184215cf None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.472699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.473338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.473616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.473886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] PATH_INFO: `/v3/projects/64c93243156f4ea7902d4e2bb9d5484f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.474980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] RBAC: Authorizing `identity:get_project(project_id=64c93243156f4ea7902d4e2bb9d5484f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.476381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83844f-2581-4b7f-84fb-241e19f17cfb None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.478017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 594/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/projects/64c93243156f4ea7902d4e2bb9d5484f => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.485307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.543351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.583522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20039499-d0e8-4e20-a03d-36d106a950c0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.616770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388d180-676e-4711-9892-0468baa3840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.617397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d180-676e-4711-9892-0468baa3840f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.617677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d180-676e-4711-9892-0468baa3840f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.617949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d180-676e-4711-9892-0468baa3840f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.618655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388d180-676e-4711-9892-0468baa3840f None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=64c93243156f4ea7902d4e2bb9d5484f, user.id=163ac0ed3e864fe79ac8b936094a0d03) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.618963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388d180-676e-4711-9892-0468baa3840f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.620415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388d180-676e-4711-9892-0468baa3840f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.625968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 606/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/role_assignments?user.id=163ac0ed3e864fe79ac8b936094a0d03&scope.project.id=64c93243156f4ea7902d4e2bb9d5484f&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:41.468638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=75 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.623427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=39 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:26.921866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c834dbc-a3a5-4903-9f59-100fffa781b2 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.922471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c834dbc-a3a5-4903-9f59-100fffa781b2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.922471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c834dbc-a3a5-4903-9f59-100fffa781b2 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.924071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 595/1201] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:26.930054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:26.930560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:26.930652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:26.952553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:26.954035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:27.045893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 607/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:27.357045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c834dbc-a3a5-4903-9f59-100fffa781b2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.498997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c834dbc-a3a5-4903-9f59-100fffa781b2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.555361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.556207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.556528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.556922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.572188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.573687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:27.594975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:27.596117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:27.597402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:27.597402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:27.597402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:27.605000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 596/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:27.605169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.729797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b1667-ea3c-4cb3-9dca-e907b10892a5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.776224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.784648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.785608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.785859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.786311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.787119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:27.787596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.790185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd15c6-f97a-4f82-a206-a8dbd4e2459e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.812927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.818130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 608/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:27.880756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.940615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071a92-e26c-4624-8338-3a109a1368ec None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.984739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.985317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.985611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.985945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.986612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.988176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3e897-3579-4fd1-9fc7-dd217025739d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:28.017006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 597/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:27 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:30.254434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2728675-d4a5-4646-8988-18b5a656fbae None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.254972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2728675-d4a5-4646-8988-18b5a656fbae None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.254972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2728675-d4a5-4646-8988-18b5a656fbae None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.256312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 609/1206] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:30.262677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.264123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.264123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.285055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:30.286359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:30.361766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 598/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:30.651226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2728675-d4a5-4646-8988-18b5a656fbae None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.782915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2728675-d4a5-4646-8988-18b5a656fbae None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.823499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.824148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.824432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.824689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.838059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:30.843839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:30.844866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:30.849947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 610/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:30.850911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.901759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:30.978639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45aa0-a15f-4bb6-966d-7a23e3c90d68 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.017746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.018359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.018641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.018917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.019576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.019880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.021278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.032310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 599/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:31.038230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.104011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.171464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241cdcb-b7eb-4baa-9c94-1ccaac64b1bb None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.211446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.212215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.212567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.212910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.225175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.227314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:31.243666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:31.244839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:31.245990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:31.245990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:31.245990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:31.245990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 611/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:31.250688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.363883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.368935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7931b7-e99e-4d0f-accc-6f6ea547bafa None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.414832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.415443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.415724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.415993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.416633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.416939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.418492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.432231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 600/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:31.438228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.566952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1a8a-c265-456a-bbad-1c9105985325 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.568562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:31.610699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.611447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.611718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.612016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.620858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.622346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:31.635536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:31.636630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:31.637733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 612/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:31.642814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.752983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98ef60-394f-4ffe-8d02-4ed484ba4e7d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.799374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.799992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.800394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.800677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:31.801558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:31.801863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:31.803579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:31.821531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 601/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:31.827455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:31.954300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ac8b9-23b1-47aa-ad5a-7420fa0b9fe4 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:31.994480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:31.995287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:31.995641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:31.996007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.028570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.080970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.085318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.209928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 613/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:32.215932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.236818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.327782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deec0-f8f1-487e-b54f-219c308d671c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.368239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.368887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.369232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.369558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.381638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.383595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:32.398889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:32.399889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:32.400837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:32.400837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:32.400837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:32.400837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 602/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:32.407293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.479009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.526466 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de126-7dcf-4eb2-99e2-7d60fdba2bdd None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.564019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.564701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.565009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.565282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.565987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.566362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.567906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.586378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 614/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.588146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.663017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.698762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117c26e-66fb-4a51-b836-c1910eb356e3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.738019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.738653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.738934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.739211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.748775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.750451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:32.765603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:32.766675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:32.767660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 603/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:32.772903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:32.894102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab627239-a32a-4ea6-a7b3-7b355e0ff6c7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:32.920263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.933276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:32.933915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.934191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.934542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.935252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:32.935565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:32.937162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78743ab-6ccd-4d3c-8800-b3921ee03765 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:32.951882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 615/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:32.958704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:33.093821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b084-df0e-44e1-8c53-5c9bb7e021a3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:33.123580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:33.130324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:33.130955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:33.131209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:33.131481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:33.132205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=be582810785a49d48b0a06cd66b316a7) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:33.132551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:33.134011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f52ba-9334-4f96-b65d-0464a8ede2d3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:33.139606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 604/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:32 2022] GET /identity/v3/role_assignments?user.id=be582810785a49d48b0a06cd66b316a7&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:35.063817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bac94-1b27-4945-91f7-569720caeabe None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.064397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bac94-1b27-4945-91f7-569720caeabe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.064397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bac94-1b27-4945-91f7-569720caeabe None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.065958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 616/1220] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:35.071848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.072280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.072447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.093848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:35.095041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:35.170888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 605/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:35.444056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bac94-1b27-4945-91f7-569720caeabe None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:35.555256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bac94-1b27-4945-91f7-569720caeabe None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:35.591073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:35.591701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.592008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.592259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.604338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:35.609486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:35.610468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:35.611308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 617/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:35.617549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:35.722327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2b539-40b5-4255-8779-30d789fb2059 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:35.749562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:35.762177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:35.763003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.763279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.763554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.764279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] RBAC: Adding query filter params (name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:35.764590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:35.766130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:35.776029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 606/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:35.782151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:35.833996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:35.899332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d8a3-bb51-4a3e-9d15-dffc52731762 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:35.938849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492903-eed6-490d-a107-5757481d9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:35.939600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492903-eed6-490d-a107-5757481d9536 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:35.940062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492903-eed6-490d-a107-5757481d9536 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:35.940522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492903-eed6-490d-a107-5757481d9536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:35.954446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02492903-eed6-490d-a107-5757481d9536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:35.955881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02492903-eed6-490d-a107-5757481d9536 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-02492903-eed6-490d-a107-5757481d9536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:35.971379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:35.972535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:35.973681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:35.973681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:35.973681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:35.973681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 618/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:35.979889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.092688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3826-8b57-4259-b26e-c4b77ad0df40 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.131059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.131669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.131952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.132858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.133729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:36.134028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.135554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.148119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 607/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:36.154131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492903-eed6-490d-a107-5757481d9536 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.255229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.262882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492903-eed6-490d-a107-5757481d9536 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.298246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.299003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.299279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.299548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.307595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.309052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:36.322988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:36.324302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:36.325308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 619/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:36.332651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.386613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.442071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27b4-59e8-4538-bbf9-229ed367067b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.478687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.479303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.479585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.479858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.480689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:36.481034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.482524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.502765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 608/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:36.509212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.620583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f8862-996f-4dfe-ad86-40a8239585b7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.654218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.654849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.655130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.655403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.670157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.671628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:36.687408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:36.688462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:36.689411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:36.689411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:36.689411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:36.689411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 620/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:36.694989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.805136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5ee8c-b860-4f1a-b857-44cb8c0df172 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.807687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.843308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.844165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.844644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.845107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.845757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:36.846095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.848227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.861994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 609/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:36.868810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.879651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.980763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39c452-98ce-45f8-ac49-a0cb3e20c8c6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.013641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.014329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.014576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.014854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.021746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.023812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:37.037835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:37.039583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:37.045426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 621/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:37.046172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.160882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320e6df-0d57-4d75-9511-26f7898c1aa3 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.196478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.197223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.197526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.197827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.198752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:37.198893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.200435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eefb84-a6b1-4b59-a5dc-fb7d41b8c111 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.213916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 610/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:37.220502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.291820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:37.338609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf488431-59d6-4704-a0b4-4a35506b7d5d None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.372790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.373469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.373745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.374012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/be582810785a49d48b0a06cd66b316a7/roles/bba68921613447a694c49e69cc04e6c9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.389037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=be582810785a49d48b0a06cd66b316a7, role_id=bba68921613447a694c49e69cc04e6c9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.390473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b41bcf-b783-4695-9ff2-70c40fad903f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.404765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 622/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:37 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/be582810785a49d48b0a06cd66b316a7/roles/bba68921613447a694c49e69cc04e6c9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:36:39.462402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581571f-4791-4b4c-aaab-3187cadac4df None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.463127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581571f-4791-4b4c-aaab-3187cadac4df None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.463127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581571f-4791-4b4c-aaab-3187cadac4df None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.464488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 611/1233] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:39.470159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.470592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.470696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.490920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:39.492714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:39.565215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 623/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:39.861352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581571f-4791-4b4c-aaab-3187cadac4df None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:39.888511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:39.980515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581571f-4791-4b4c-aaab-3187cadac4df None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.020451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.021293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.021785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.022133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.034707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:40.040147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:40.041339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:40.042395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 612/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Wed Aug 31 09:36:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:40.048071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.101844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.162772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a98a8c-e6ea-489d-9a4b-45dc5d56c1e3 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.204455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.205099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.205379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.205653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.206314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] RBAC: Adding query filter params (name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:40.206619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.207993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.217464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 624/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.223814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.316175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.336787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248e2a3-03d6-4324-9bf6-e044eff020ac None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.379581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.380231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.380586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.380875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.393470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.394921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:40.411487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:40.412729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:40.414071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:40.414071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:40.414071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:40.414071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 613/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:40.418759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.537696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289941-98ca-4246-94a7-b655f300e00c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.547694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.575474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.576108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.576398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.576751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.577440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:40.577792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.579335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.592072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 625/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.598162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.707679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17b362-6db4-4eaf-9580-a702054cb423 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.744892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.745557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.745838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.746152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.749126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.754798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.756645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:40.770419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:40.771534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:40.772928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 614/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:40.777890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.887857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a941b-cf69-4eb3-a6d1-7fe51d04e9c8 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.930364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.930975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.931325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.931600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.932349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:40.932656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.934169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.958737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 626/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.965243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.075837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977bc9b-3d33-465c-8f57-acb329a074ff None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.118178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.118826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.119172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.119487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.186553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:41.194301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.195947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.307864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 615/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:41.310877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.347776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:41.432013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818bc0-0c12-417c-b8bb-38e62211a94c None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.470830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.471524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.471837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.472112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.484224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.485865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:41.502215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:41.503534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:41.504779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:41.504779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:41.504779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:41.504779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 627/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:41.509571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.612815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:41.619110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f29e-44f3-4eb4-be26-3dc1a5891bb5 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.660431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.661091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.661376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.661654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.662305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:41.662612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.664696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.676781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 616/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:41.684030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.773328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:41.797241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23621e0a-65a3-40c7-ac12-2af0107b26f6 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:41.831205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:41.831839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:41.832118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:41.832401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:41.840495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:41.842150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:41.859470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:41.860615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:41.861770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 628/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:41.869871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:41.994526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d10e04-3ff9-4623-9b7f-963f073d6239 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:42.036655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:42.037366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:42.037702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:42.038006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:42.039496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:42.039850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:42.041473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1ed9-2901-4d65-b6d3-677c337ae768 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:42.056651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 617/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:42.062780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:42.182233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166016d-b10a-4f84-9a11-670af39c74c0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:42.220432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:42.221077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:42.221357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:42.221640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:42.222371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] RBAC: Adding query filter params (role.id=bba68921613447a694c49e69cc04e6c9, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=be582810785a49d48b0a06cd66b316a7) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:42.222681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:42.224125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77986a-93b2-4ebc-8b07-d6885a1feb7d None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:42.231213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 629/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:42 2022] GET /identity/v3/role_assignments?user.id=be582810785a49d48b0a06cd66b316a7&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=bba68921613447a694c49e69cc04e6c9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:44.373896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399baa-e604-48f6-aab0-cd3bdaabd03e None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.374415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399baa-e604-48f6-aab0-cd3bdaabd03e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.374415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399baa-e604-48f6-aab0-cd3bdaabd03e None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.375872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 618/1247] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:44.381609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.382037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.382168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.390712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:44.405896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:44.407091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:44.490929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 630/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:44.767692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399baa-e604-48f6-aab0-cd3bdaabd03e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:44.787168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:44.876746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399baa-e604-48f6-aab0-cd3bdaabd03e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:44.916406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:44.917096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:44.917408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:44.917644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:44.928144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:44.934391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:44.935541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:44.938201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 619/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:44.941648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:45.059673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c0bdc-c0e2-4091-a2e5-84387c8ff55a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:45.104897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:45.111041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:45.111793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:45.112078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:45.112390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:45.113086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:45.113398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:45.114793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:45.123162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 631/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:45.129119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:45.227968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:45.255192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b513-c584-48bf-b3b1-57a1c5a281ae None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:45.303828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:45.304608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:45.304936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:45.305300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:45.318881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:45.320890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:45.341763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:45.343375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:45.344514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:45.344514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:45.344514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:45.346052 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 620/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:45.353683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:45.474481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1205a81-71c6-4c04-ba52-7d32da6cbd02 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:45.512362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:45.513083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:45.513370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:45.513739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:45.514457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:45.514767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:45.516220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:45.530870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 632/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:45.536595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:45.645937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e149-dd72-4516-8f01-033a562f4d84 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:45.674711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:45.684849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:45.685614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:45.685949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:45.686283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:45.696240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:45.698617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:45.713904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:45.715249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:45.716622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 621/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:45.721410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:45.785666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:45.834975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38fe00-cf6a-4c2f-890a-eca84be46731 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:45.875346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:45.875982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:45.876267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:45.876536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:45.877296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:45.877600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:45.879027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:45.899901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 633/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:45.906460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.022919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610a66-9a71-46d3-a69e-7d597239795e None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:46.060544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:46.061233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.061584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.061920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.127951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:46.143776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:46.145426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:46.262785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 622/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:46.269191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.400853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0753-d495-4de7-a64e-6790751c4049 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:46.414837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:46.436826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:46.437520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.437824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.438098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.450612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:46.452088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:46.468325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:46.469554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:46.470666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:46.470666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:46.470666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:46.470666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 634/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:46.475545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.567334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:46.580800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f31f10-53a6-48fc-9f7e-ef60c79f1234 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:46.617056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:46.617673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.617960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.618231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.618882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:46.619185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:46.620569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:46.633643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 623/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:46.640095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.745290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafb918-63b0-48eb-9809-82b09fe9fe1e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:46.778882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:46.779515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.779800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.780076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.786803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:46.788258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:46.799495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:46.800441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:46.801410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 635/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:46.807325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.911865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0283-8242-4c20-9fdb-af0571d2a5da None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:46.946186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:46.946794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:46.947077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:46.947347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:46.948062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:46.948369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:46.949820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc61f3-ec21-4daa-95d2-f5725ab4a3a2 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:46.962673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 624/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:46.968763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:46.994979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:47.066463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c441e7-a88d-40c4-b006-e86682295d66 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:47.101766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:47.102400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:47.102684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:47.102959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:47.103677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=be582810785a49d48b0a06cd66b316a7) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:47.103985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:47.105457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb580470-5a8c-4a78-8acd-4769b0f80dd7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:47.110671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 636/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:46 2022] GET /identity/v3/role_assignments?user.id=be582810785a49d48b0a06cd66b316a7&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:49.114097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d83c-e627-4fa1-9a9b-2a4c07721010 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.114615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d83c-e627-4fa1-9a9b-2a4c07721010 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.114747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d83c-e627-4fa1-9a9b-2a4c07721010 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.115983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 625/1261] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:49.121131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.121581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.121704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.139924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:49.141123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:49.204794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 637/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:49.469263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66d83c-e627-4fa1-9a9b-2a4c07721010 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:49.480177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:49.569635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66d83c-e627-4fa1-9a9b-2a4c07721010 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:49.609517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:49.610136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.610415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.610689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.621183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:49.625926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:49.627702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:49.629397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 626/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:49.633728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:49.651410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:49.731860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e81904-d74e-427d-b9ec-8abfc8df1e1d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:49.766453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:49.767050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.767329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.767599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.768252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:49.768555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:49.769948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:49.777706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 638/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:49.783469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:49.852664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:49.888331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62543e6f-a98a-4fea-8114-520df40bb008 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:49.927368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:49.928019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:49.928297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:49.929152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:49.940057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:49.941792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:49.957107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:49.958274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:49.959285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:49.959285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:49.959285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:49.959285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 627/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:49.965300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.046396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:50.065590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc43a-dad3-494d-befc-954d453f19c9 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.098467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.099074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.099356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.099630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.100769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:50.101109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.102476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.113911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 639/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:50.120243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.216753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943a2dd-aeb5-42f7-937a-c83673df1aff None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.235999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:50.250259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.250872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.251151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.251423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] PATH_INFO: `/v3/users/placement` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.258696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.260124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:50.271094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:50.272722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:50.273753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 628/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:50.278729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.387625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebaea2-f72b-408e-9efe-82e8509b1793 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.421113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.421801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.422125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.422412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.423250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:50.423607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.425276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.434797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:50.444571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 640/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:50.450269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.549759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf02f-41ff-4a79-9f4b-2d21d99d521b None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.581683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.582293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.582572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.582846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.594412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.595827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:50.613699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:50.614841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:50.615874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:50.615874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:50.615874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:50.615874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 629/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:50.621997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.724147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76863371-7bf6-4a45-9f5d-c31477df9da7 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.756918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.757558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.757838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.758104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.758725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:50.759025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.760384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.771125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 641/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:50.777355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.807920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:50.875691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209dbc-31e2-42b7-925d-735efbd7ea8c None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:50.908572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:50.909276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:50.909556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:50.909825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:50.917185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:50.918608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:50.929611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:50.930667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:50.931941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 630/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:50.936524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:50.999187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:51.039270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108fab-24b4-41a9-96b5-53d6ecd833c0 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:51.072211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:51.072819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:51.073127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:51.073400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:51.074082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:51.074384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:51.075823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959db4cc-1514-4d85-97a2-6ef631162010 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:51.087800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 642/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:51.093811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:51.183394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:51.192261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770271-e5ff-4dce-b473-52c5710d50af None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:51.224784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:51.225561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:51.225844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:51.226118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] PATH_INFO: `/v3/projects/6a25e6d820824e73b621c2c157e75383/users/be582810785a49d48b0a06cd66b316a7/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:51.228000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a25e6d820824e73b621c2c157e75383, user_id=be582810785a49d48b0a06cd66b316a7, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:51.229454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6c6ef-6c6b-4b38-a734-861503e12c12 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:51.242648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 631/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:51 2022] PUT /identity/v3/projects/6a25e6d820824e73b621c2c157e75383/users/be582810785a49d48b0a06cd66b316a7/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:53.129868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde5b5-8a04-4213-ab64-6dcfd13cd89a None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.130348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde5b5-8a04-4213-ab64-6dcfd13cd89a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.130441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efde5b5-8a04-4213-ab64-6dcfd13cd89a None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.132250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 643/1274] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:53.137302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.137713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.137870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.156290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:53.157506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:53.221408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 632/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:53.493467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efde5b5-8a04-4213-ab64-6dcfd13cd89a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.592493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efde5b5-8a04-4213-ab64-6dcfd13cd89a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.609637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.625962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.626583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.626860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.627129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.637718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:53.642498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:53.643633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:53.644539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 644/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:53.650234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.729165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:53.747220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04407c-c5b4-4615-8f1f-be5b6683b917 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.780707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.781341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.781625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.781907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.782565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:53.782869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.784219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.792940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 633/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:53.799261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:53.898407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d965c7f-d25a-475d-92d7-85fa705f9416 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:53.931250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:53.931895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:53.932177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:53.932454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:53.942288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:53.944488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:53.958806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:53.960473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:53.961643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:53.961643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:53.961643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:53.961643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 645/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:53.966814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:54.063820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8855a35-31c4-4523-a6fb-aa4a253664d8 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:54.098749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:54.099354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:54.099634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:54.099908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:54.100584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:54.100892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:54.102263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:54.113722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 634/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:54.119991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:54.150371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:54.219659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab8be-f666-4240-a8f5-5ecf7fa81797 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:54.252318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:54.252940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:54.253251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:54.253523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:54.261303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:54.262736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:36:54.273733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:54.274828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:54.275796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 646/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:54.280797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:54.284631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:54.386727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717befa-2709-4b79-9eec-f92190e0bf4d None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:54.421952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:54.422594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:54.422889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:54.423214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:54.423996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:54.424304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:54.425873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:54.445407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 635/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Wed Aug 31 09:36:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:54.451341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:54.544916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:54.556018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb051978-d163-4f7d-8b5e-6acd8d84fe6e None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:54.590511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:54.591169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:54.591497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:54.591787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:54.661849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:54.663925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:54.754282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:54.765577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 647/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:54.771471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:54.836361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:54.876962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076fa49a-5e69-4262-84b3-2b5a6a44d0ee None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:54.911417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:54.912141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:54.912436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:54.913335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:54.923770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:54.925413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:54.940884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:54.941991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:36:54.943551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:54.943551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:36:54.943551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:54.943551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 636/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:54.948442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:55.052933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8754b-725e-4730-b479-8e252d5fedf2 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:55.088613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:55.089181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.089470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.089750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.090445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:55.090755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:55.092130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:55.103851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 648/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Wed Aug 31 09:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:55.109834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:55.207406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25e42-8d5c-4b33-84cc-473c4981c807 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:55.221196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:55.241177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:55.241794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.242078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.242355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.248766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:55.250226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:36:55.261742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:55.262806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: ERROR keystone.server.flask.application Aug 31 09:36:55.263943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 637/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Wed Aug 31 09:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:55.268715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:55.320703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:55.374313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850390f6-f733-4498-a956-e7ef4244bd91 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:55.407757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:55.408357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.408639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.408911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.409618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:55.409924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:55.411356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e7f99-d3bc-4650-b848-f720f3284c26 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:55.423071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 649/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Wed Aug 31 09:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:55.429158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:55.530259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fac8a5-5804-467e-a17b-3f3be4652604 None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:55.563145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:55.563742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:55.564026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:55.564303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:55.565045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] RBAC: Adding query filter params (role.id=3058b83cad264d36bc1ba356d7561bef, scope.project.id=6a25e6d820824e73b621c2c157e75383, user.id=be582810785a49d48b0a06cd66b316a7) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:55.565359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:55.566779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd981f4-16cd-42fe-a27d-cb7c208801db None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:55.573305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 638/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:55 2022] GET /identity/v3/role_assignments?user.id=be582810785a49d48b0a06cd66b316a7&scope.project.id=6a25e6d820824e73b621c2c157e75383&role.id=3058b83cad264d36bc1ba356d7561bef => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:57.503496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d935-1505-41f9-92f5-92286aaa4b45 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:57.504032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d935-1505-41f9-92f5-92286aaa4b45 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:57.504032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d935-1505-41f9-92f5-92286aaa4b45 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:57.505401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 650/1288] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:57.511099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:57.511520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:57.511661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:57.530228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:57.531417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:57.568489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:57.608779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 639/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:57.892382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836d935-1505-41f9-92f5-92286aaa4b45 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:57.991346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836d935-1505-41f9-92f5-92286aaa4b45 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:58.001574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:58.024899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:58.025546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:58.025824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:58.026094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:58.026665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:58.028033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:58.036626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:58.037921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:36:58.039013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 651/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:36:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:58.043812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:58.146580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf8e71-6f17-4bff-98a5-d70f0e9d8741 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:58.179613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:58.180214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:58.180495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:58.180777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:58.181425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:58.181728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:58.183088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4b8e2-2071-4475-9ab0-98048ab70af4 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:58.187790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 640/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 31 09:36:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:58.194488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:58.291298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdaaf00-a1ab-4ed0-bdc5-da65579b0598 None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:58.327000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:58.327826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:58.328325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:58.328769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] PATH_INFO: `/v3/services` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:58.329732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] RBAC: Adding query filter params (type=placement) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:58.330225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:58.332238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea0449-c4c5-4da6-a571-9832c99cfbba None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:58.339021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 652/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 31 09:36:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:00.274802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98cacf-d84a-4a06-bbd6-c9cadd777f34 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:00.275346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98cacf-d84a-4a06-bbd6-c9cadd777f34 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:00.275346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98cacf-d84a-4a06-bbd6-c9cadd777f34 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.276696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 641/1293] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:00.281728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d077de-8ccf-4d38-b859-f9c0020b38e0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:00.282117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d077de-8ccf-4d38-b859-f9c0020b38e0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:00.282277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d077de-8ccf-4d38-b859-f9c0020b38e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.301247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f6d077de-8ccf-4d38-b859-f9c0020b38e0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:00.302450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f6d077de-8ccf-4d38-b859-f9c0020b38e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:00.332582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:00.371339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 653/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:00.642284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98cacf-d84a-4a06-bbd6-c9cadd777f34 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:00.673122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:00.744582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98cacf-d84a-4a06-bbd6-c9cadd777f34 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:00.778017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:00.778610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:00.778888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:00.779159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:00.779805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:00.781175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bd22e-b9a7-4f96-ba65-92ca26ca0e56 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:00.794940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 642/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Wed Aug 31 09:37:00 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:02.704294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf136219-21af-4f15-ab8c-9797cf93aaae None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:02.704860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf136219-21af-4f15-ab8c-9797cf93aaae None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:02.704860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf136219-21af-4f15-ab8c-9797cf93aaae None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:02.706210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 654/1296] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:02.711670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:02.712106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:02.712208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:02.730527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:02.731772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:02.803554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 643/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:03.077447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf136219-21af-4f15-ab8c-9797cf93aaae None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.177071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf136219-21af-4f15-ab8c-9797cf93aaae None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.210482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.211105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.211386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.211665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.212235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.213639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:03.222300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:03.223561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:03.225302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 655/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:03.229962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.248259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.336710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18653c1a-6362-4b37-af8e-b3db2d1a5649 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.378110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.378830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.379126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.379393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.380034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] RBAC: Adding query filter params (name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:03.380332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.382269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.387556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 644/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:03.394338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.426922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.495728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a20d-4749-4835-a750-3f0b6452db7f None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.532156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.532763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.533079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.533351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.534046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] RBAC: Adding query filter params (interface=public, service_id=d60a6f34be5542838f4e9bd037d06a64, region_id=RegionOne) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:03.534353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.535723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cfbde-0d20-461c-8cd0-3d4835d87ced None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.540741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 656/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/endpoints?service_id=d60a6f34be5542838f4e9bd037d06a64&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:03.546503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:03.629859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:03.649377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016525ab-4701-4528-ac84-6b54bbf3fdee None admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:03.682007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:03.682605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:03.682881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:03.683151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:03.683745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:03.685251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33a6a1-3ecc-4f8d-a7bd-42214136aa28 None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:03.691863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 645/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Wed Aug 31 09:37:03 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:37:05.638455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540232e-97ce-4ebe-ad97-97332c138a92 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:05.638986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540232e-97ce-4ebe-ad97-97332c138a92 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:05.638986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540232e-97ce-4ebe-ad97-97332c138a92 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:05.640325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 657/1302] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:05.645260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:05.645705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:05.645811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:05.664089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:05.665310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:05.730304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 646/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:06.004180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540232e-97ce-4ebe-ad97-97332c138a92 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:06.097575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:06.105087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540232e-97ce-4ebe-ad97-97332c138a92 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:06.137112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:06.137735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:06.138016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:06.138291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] PATH_INFO: `/v3/services/placement` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:06.138861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:06.140227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:06.149909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:06.151106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: ERROR keystone.server.flask.application Aug 31 09:37:06.152142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 658/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Wed Aug 31 09:37:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:06.156947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:06.235404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:06.257444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e0bbc-469d-4547-8644-4db5bb91b45a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:06.290879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:06.291475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:06.291753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:06.292019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] PATH_INFO: `/v3/services` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:06.292640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] RBAC: Adding query filter params (name=placement) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:06.292937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:06.294300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c1b4a-dfbb-4842-9fe0-5d24795c6f5e None admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:06.299242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 647/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Wed Aug 31 09:37:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:06.305346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:06.411698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3a48a-3a15-4823-b953-61d71e6e35cb None admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:06.444747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:06.445392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:06.445675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:06.445950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:06.446615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:06.447977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422efec4-fc00-4e2c-99bd-b13f4ffa19ac None admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:06.466971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 659/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:37:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:16.296473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=30 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:16.464201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:09.586180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7aa2-279b-4b6a-9f1f-e4182471a22a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:09.587235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7aa2-279b-4b6a-9f1f-e4182471a22a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:09.587735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7aa2-279b-4b6a-9f1f-e4182471a22a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:09.594631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45028-c8d3-475d-a2f9-b24f3e1427cb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:09.595528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45028-c8d3-475d-a2f9-b24f3e1427cb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:09.596012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e45028-c8d3-475d-a2f9-b24f3e1427cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:09.601913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 648/1307] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:09 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:09.675809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a6e45028-c8d3-475d-a2f9-b24f3e1427cb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:09.677398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a6e45028-c8d3-475d-a2f9-b24f3e1427cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:09.813363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 660/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:10.113082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:10.113597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:10.114063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:10.115575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 649/1309] 10.222.0.25 () {54 vars in 965 bytes} [Wed Aug 31 09:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:10.122082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf285-29ec-4288-b131-268157a6dfe8 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:10.122609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf285-29ec-4288-b131-268157a6dfe8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:10.123088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caf285-29ec-4288-b131-268157a6dfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:10.276365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f8caf285-29ec-4288-b131-268157a6dfe8 None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:10.277608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f8caf285-29ec-4288-b131-268157a6dfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:10.325835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:10.435656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 661/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Aug 31 09:38:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:10.450116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:10.452196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:10.618219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11e37-c823-49cc-ab49-dc8c6ee06621 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:10.704573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:10.705499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:10.706487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:10.706924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:10.708741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:10.769928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:10.828975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:10.830423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ff21-8dde-4615-b9dd-d4d1344eb138 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:10.850586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 650/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:13.086602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:13.087138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:13.087138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:13.088782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 662/1312] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:13.094550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41254b5f-7a75-4309-bc47-e879fd4c91b0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:13.095041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41254b5f-7a75-4309-bc47-e879fd4c91b0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:13.095132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41254b5f-7a75-4309-bc47-e879fd4c91b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:13.178552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-41254b5f-7a75-4309-bc47-e879fd4c91b0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:13.179772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-41254b5f-7a75-4309-bc47-e879fd4c91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:13.284735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 651/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:13.572529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:13.574105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:13.678316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a876672-141f-492a-ba4b-8e5fdd584acf None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:13.748096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:13.750675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:13.751296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:13.751763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:13.752074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:13.754698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:13.862793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:13.864243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0e75-cf9c-418b-b3f8-b6e908400a71 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:13.881768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 663/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:16.103376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:16.103869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:16.103974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:16.105299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 652/1315] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:16.110422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfdd9-e57e-4db9-a497-3684f1543698 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:16.110829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfdd9-e57e-4db9-a497-3684f1543698 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:16.110968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acfdd9-e57e-4db9-a497-3684f1543698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:16.176221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-94acfdd9-e57e-4db9-a497-3684f1543698 None None] MFA Rules not processed for user `a48ab2dbb7ac436d8fc857a199e4e839`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:16.177458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-94acfdd9-e57e-4db9-a497-3684f1543698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:16.227232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:16.271121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 664/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:16.569093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:16.570671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:16.580984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:16.676182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7768cbb-ca76-405f-944d-d44a36c9d752 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:16.748700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:16.749331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:16.749616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:16.749889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:16.751487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:16.813392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:16.860256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:16.862322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9ab38-6ff4-438b-b06b-450186bfa5d0 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:16.879768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 653/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:19.295365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.295896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.295896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.297300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 665/1318] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:19.302966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a5db-b5e2-43c0-901d-8b244d881a4a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.303395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a5db-b5e2-43c0-901d-8b244d881a4a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.303527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a5db-b5e2-43c0-901d-8b244d881a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.369174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-22d5a5db-b5e2-43c0-901d-8b244d881a4a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:19.370396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-22d5a5db-b5e2-43c0-901d-8b244d881a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:19.465942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 654/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:19.752860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:19.754417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:19.858082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564aa95-89d2-4c4f-ae11-19f6e03ed8d8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:19.929926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:19.930522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:19.930798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:19.931065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:19.932628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:19.940825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:20.038906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:20.040527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971e9fe-f133-48bf-9268-be775096ef45 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:20.057732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 666/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:22.597406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:22.597943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:22.597943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:22.599293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 655/1321] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:22.604447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce135c40-b4cb-440c-859d-d6367ad23b7c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:22.604885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce135c40-b4cb-440c-859d-d6367ad23b7c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:22.605034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce135c40-b4cb-440c-859d-d6367ad23b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:22.671210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ce135c40-b4cb-440c-859d-d6367ad23b7c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:22.672397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ce135c40-b4cb-440c-859d-d6367ad23b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:22.734538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:22.776237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 667/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:23.058609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:23.061336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:23.084631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:23.175506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e111c-0545-40a7-9e62-f48f33c5533d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:23.253665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:23.254280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:23.254563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:23.254838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:23.256430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:23.327729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:23.367865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:23.369450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb44ea-8d43-4884-998a-98d26385700b service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:23.388844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 656/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:25.367606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:25.368031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:25.368212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:25.370377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 668/1324] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:25.374949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dcfd-1607-4939-8aec-9e9d802121f6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:25.375364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dcfd-1607-4939-8aec-9e9d802121f6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:25.375495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dcfd-1607-4939-8aec-9e9d802121f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:25.441319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3149dcfd-1607-4939-8aec-9e9d802121f6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:25.442518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3149dcfd-1607-4939-8aec-9e9d802121f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:25.538094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 657/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:25.819098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:25.820699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:25.922879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a202fc-1954-48b6-bee3-fcd205bfdd10 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:26.013357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:26.014001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:26.014283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:26.014558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:26.016146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:26.035468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:26.132700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:26.134174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706990d6-1fcc-4128-9605-814369b91485 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:26.152682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 669/1326] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:28.120836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:28.121343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:28.121437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:28.122825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 658/1327] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:28.128527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa328e3e-15ce-495d-9507-0a1ee365763b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:28.129011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa328e3e-15ce-495d-9507-0a1ee365763b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:28.129121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa328e3e-15ce-495d-9507-0a1ee365763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:28.199575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-aa328e3e-15ce-495d-9507-0a1ee365763b None None] MFA Rules not processed for user `a48ab2dbb7ac436d8fc857a199e4e839`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:28.200831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-aa328e3e-15ce-495d-9507-0a1ee365763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:28.266658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:28.299433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 670/1328] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:38:28.585127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:28.586762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:28.626217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:28.708406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fc74c-7271-45cd-a1c9-fde6203077b6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:28.783770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:28.784380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:28.784664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:28.784943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:28.786576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:28.865506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:28.896547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:28.898020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0512d-a23a-49cd-bf69-6a06f78ccc73 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:28.915950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 659/1329] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:31.065194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.065981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.066101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.067361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 671/1330] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:31.074525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4fc48-4f48-4ce4-b037-ee41ee8d5d10 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.074996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4fc48-4f48-4ce4-b037-ee41ee8d5d10 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.075090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4fc48-4f48-4ce4-b037-ee41ee8d5d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.140402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cdd4fc48-4f48-4ce4-b037-ee41ee8d5d10 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:31.141626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cdd4fc48-4f48-4ce4-b037-ee41ee8d5d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:31.237134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 660/1331] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:31.518108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:31.519640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:31.628361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f167a-1162-47ef-8b6f-1f4ea09180e8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:31.700357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:31.701365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:31.701650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:31.701926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:31.703527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:31.729683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:31.818652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:31.820100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca0037-a5f5-47b8-a51f-d3b87a0bfca9 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:31.838679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 672/1332] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:33.817199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:33.817736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:33.817736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:33.819092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 661/1333] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:33.824152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edfae-3252-442e-8bf2-1c6a5592a057 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:33.824535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edfae-3252-442e-8bf2-1c6a5592a057 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:33.824709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edfae-3252-442e-8bf2-1c6a5592a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:33.889790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-990edfae-3252-442e-8bf2-1c6a5592a057 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:33.890987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-990edfae-3252-442e-8bf2-1c6a5592a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:33.964363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:33.987560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 673/1334] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:34.276742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:34.278344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:34.319585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:34.383411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e316cb5-6e4b-43ad-9e70-bcc173f1e8d0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:34.454122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748992c6-ee41-400c-8320-884241545585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:34.455277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992c6-ee41-400c-8320-884241545585 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:34.455570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992c6-ee41-400c-8320-884241545585 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:34.455824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748992c6-ee41-400c-8320-884241545585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:34.457443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-748992c6-ee41-400c-8320-884241545585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:34.541586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:34.568377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748992c6-ee41-400c-8320-884241545585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:34.569860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748992c6-ee41-400c-8320-884241545585 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:34.587736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 662/1335] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:36.810687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a474-1e95-42f4-8667-19e5f90556b9 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.811211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a474-1e95-42f4-8667-19e5f90556b9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.811211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a474-1e95-42f4-8667-19e5f90556b9 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.812458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 674/1336] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:36.818501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827c261-7416-45e4-b297-40633995214d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:36.818887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827c261-7416-45e4-b297-40633995214d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:36.819076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827c261-7416-45e4-b297-40633995214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:36.884435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8827c261-7416-45e4-b297-40633995214d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:36.885675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8827c261-7416-45e4-b297-40633995214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:36.981083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 663/1337] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:37.279366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:37.279654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:37.279931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:37.281291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 675/1338] 10.222.0.25 () {54 vars in 965 bytes} [Wed Aug 31 09:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:37.287541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044431-ffdb-4495-905d-f47f64e43773 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:37.287832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044431-ffdb-4495-905d-f47f64e43773 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:37.288111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4044431-ffdb-4495-905d-f47f64e43773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:37.347651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:37.371076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a4044431-ffdb-4495-905d-f47f64e43773 None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:37.372284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a4044431-ffdb-4495-905d-f47f64e43773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:37.446239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 664/1339] 10.222.0.25 () {58 vars in 1062 bytes} [Wed Aug 31 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:37.455641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:37.457207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:37.557150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf405d-5648-4fc9-9ef1-bba156f67b35 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:37.629428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:37.629984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:37.630264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:37.630536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:37.632121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:37.667806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:37.740828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:37.742452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd843f5-eecb-467d-bdc3-75eb6001e210 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:37.759978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 676/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:42.240577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:42.241099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:42.241099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:42.242429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 665/1341] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:42.247540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677ac13-c2cc-40c1-b5b8-efb48654c5f0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:42.248005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677ac13-c2cc-40c1-b5b8-efb48654c5f0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:42.248133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677ac13-c2cc-40c1-b5b8-efb48654c5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:42.316838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a677ac13-c2cc-40c1-b5b8-efb48654c5f0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:42.318072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a677ac13-c2cc-40c1-b5b8-efb48654c5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:42.408043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:42.420183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 677/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:42.702430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:42.704651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:42.807727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e9f83-d13b-48e6-9c62-a984e2e914fa None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:42.846176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:42.883254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:42.883866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:42.884145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:42.884418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:42.886032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:42.995808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:42.997183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd83df-6606-4440-8c0b-fa5d29002468 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:43.015347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 666/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:45.569441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.570008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.570008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.571333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 678/1344] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:45.576435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872690ce-c6d0-46a7-9248-87d73696af76 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:45.576846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872690ce-c6d0-46a7-9248-87d73696af76 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:45.577144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872690ce-c6d0-46a7-9248-87d73696af76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:45.644462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-872690ce-c6d0-46a7-9248-87d73696af76 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:45.645632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-872690ce-c6d0-46a7-9248-87d73696af76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:45.664080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:45.744264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 667/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:38:46.026362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:46.027901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:46.134346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3a8dc-4412-4002-8790-d18c05afc9a9 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:46.224106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:46.224713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:46.225026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:46.225304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:46.226892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:46.270877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:46.342058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:46.343696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71bcd7-dd07-4b34-84df-546d0adb73cc service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:46.365153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 679/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:50.287170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.287960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.288408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.290271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 668/1347] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:50.295547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369d8-e1ee-47f2-91ad-767bcc4a30f7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.295972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369d8-e1ee-47f2-91ad-767bcc4a30f7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.296139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88369d8-e1ee-47f2-91ad-767bcc4a30f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.372214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c88369d8-e1ee-47f2-91ad-767bcc4a30f7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:50.373481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c88369d8-e1ee-47f2-91ad-767bcc4a30f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:50.475268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.478398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 680/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:38:50.768115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:50.769785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:50.874937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1b2f4-c11d-4604-828a-73fae1495d23 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:50.925335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:50.952985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:50.953656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:50.953927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:50.954200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:50.955790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:51.066040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:51.067394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b643a-6f72-438a-94df-e40a504c1e42 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:51.085631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 669/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:38:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:38:55.974317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa80d26-97f3-415c-ad59-b145c3e6c40e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:55.975169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa80d26-97f3-415c-ad59-b145c3e6c40e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:55.976659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa80d26-97f3-415c-ad59-b145c3e6c40e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:55.978609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 681/1350] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:38:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:55.983966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabdfd7-9946-464f-b84a-daecdcd0bd92 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:55.984988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabdfd7-9946-464f-b84a-daecdcd0bd92 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:55.985144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabdfd7-9946-464f-b84a-daecdcd0bd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:56.113357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1fabdfd7-9946-464f-b84a-daecdcd0bd92 None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:56.114959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1fabdfd7-9946-464f-b84a-daecdcd0bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:56.149916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:56.250761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 670/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:38:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:00.591309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96bb15-2aeb-4e64-912d-7a752526d39e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:00.591906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96bb15-2aeb-4e64-912d-7a752526d39e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:00.591906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96bb15-2aeb-4e64-912d-7a752526d39e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:00.593863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 682/1352] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:00.598618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbf98-1c9f-4f7e-a8e4-1cab52752a83 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:00.599113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbf98-1c9f-4f7e-a8e4-1cab52752a83 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:00.599384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbf98-1c9f-4f7e-a8e4-1cab52752a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:00.679056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b1acbf98-1c9f-4f7e-a8e4-1cab52752a83 None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:00.680634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b1acbf98-1c9f-4f7e-a8e4-1cab52752a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:00.760391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:00.800406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 671/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:39:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:02.150286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ec1e-066e-486c-9e73-46472d994368 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:02.150805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ec1e-066e-486c-9e73-46472d994368 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:02.150901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255ec1e-066e-486c-9e73-46472d994368 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:02.152153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 683/1354] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:02.157829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2cf8-25f2-47e2-b4e8-bce944f8e09f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:02.158242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2cf8-25f2-47e2-b4e8-bce944f8e09f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:02.158372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2cf8-25f2-47e2-b4e8-bce944f8e09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:02.235254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8f5f2cf8-25f2-47e2-b4e8-bce944f8e09f None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:02.236440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8f5f2cf8-25f2-47e2-b4e8-bce944f8e09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:02.345386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 672/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:39:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:05.595682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa744a81-c598-49bc-af4e-720b79e1f6b2 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:05.596274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa744a81-c598-49bc-af4e-720b79e1f6b2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:05.596274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa744a81-c598-49bc-af4e-720b79e1f6b2 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:05.597586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 684/1356] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:05.602748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9f9d-9927-436a-8740-0256c16e5b72 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:05.603148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9f9d-9927-436a-8740-0256c16e5b72 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:05.603343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed9f9d-9927-436a-8740-0256c16e5b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:05.661019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:05.703153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-06ed9f9d-9927-436a-8740-0256c16e5b72 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:05.704282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-06ed9f9d-9927-436a-8740-0256c16e5b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:05.811007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 673/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:06.113949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:06.114455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:06.114883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:06.116603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 685/1358] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 31 09:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:06.124196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387310-a2a1-43af-a143-4e11dd71781a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:06.125179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387310-a2a1-43af-a143-4e11dd71781a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:06.125608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387310-a2a1-43af-a143-4e11dd71781a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:06.212913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e8387310-a2a1-43af-a143-4e11dd71781a None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:06.214137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e8387310-a2a1-43af-a143-4e11dd71781a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:06.265223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:06.351266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 674/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 31 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:06.359774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:06.361480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:06.536638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79232d6-b403-4aa5-84a3-a07cb67b41dc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:06.613115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:06.654489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:06.655092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:06.655939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:06.656217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:06.657829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:06.749913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:06.751558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50afee8-811b-4181-9dec-6e5751bca3a4 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:06.770962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 686/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:08.889329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:08.890444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:08.890444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:08.891928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 675/1361] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:08.896949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715dc8-0b0c-400c-aced-0c52e1a8d891 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:08.897377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715dc8-0b0c-400c-aced-0c52e1a8d891 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:08.897517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715dc8-0b0c-400c-aced-0c52e1a8d891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:08.965129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e2715dc8-0b0c-400c-aced-0c52e1a8d891 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:08.966334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e2715dc8-0b0c-400c-aced-0c52e1a8d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:09.006526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:09.089679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 687/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:09.372635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:09.374803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:09.548207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243ef49-2e38-46f6-88b3-1b072420bcf9 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:09.558881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:09.685199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:09.686044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:09.687053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:09.687504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:09.689555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:09.794097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:09.795604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58c5ac-f0c0-45d5-a844-280d4b83e5dd service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:09.817034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 676/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:12.060531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0de932-4c78-40d8-bdbc-592e5fb8b585 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:12.061245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0de932-4c78-40d8-bdbc-592e5fb8b585 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:12.061245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0de932-4c78-40d8-bdbc-592e5fb8b585 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:12.062387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 688/1364] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:12.068019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b3de-ea59-4976-8de6-c454c83d125f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:12.068459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b3de-ea59-4976-8de6-c454c83d125f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:12.068620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b3de-ea59-4976-8de6-c454c83d125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:12.089727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:12.144257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6379b3de-ea59-4976-8de6-c454c83d125f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:12.145505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6379b3de-ea59-4976-8de6-c454c83d125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:12.248694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 677/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:12.542978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:12.543582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:12.543931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:12.545364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 689/1366] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 31 09:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:12.550912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44b1b0-6095-4a25-a0d9-07ed4c4c1461 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:12.551211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44b1b0-6095-4a25-a0d9-07ed4c4c1461 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:12.551492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44b1b0-6095-4a25-a0d9-07ed4c4c1461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:12.636325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3b44b1b0-6095-4a25-a0d9-07ed4c4c1461 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:12.637716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3b44b1b0-6095-4a25-a0d9-07ed4c4c1461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:12.676080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:12.769318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 678/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 31 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:12.776234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:12.777770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:12.943965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233d9e7-8b13-402c-8c6b-87039c11485c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:12.965637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:13.067133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:13.067816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:13.068106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:13.068304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:13.069989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:13.174371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:13.176461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dfc8e-3cdf-491a-a270-e46a13d34b24 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:13.196807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 690/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:39:13.731082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e666ce-6768-41f3-acbc-ccb00c6d6d9b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:13.731619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e666ce-6768-41f3-acbc-ccb00c6d6d9b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:13.731898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e666ce-6768-41f3-acbc-ccb00c6d6d9b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:13.734175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 679/1369] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:13.739278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc2f9-9129-4e01-8086-86a32fcc3cfb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:13.739564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc2f9-9129-4e01-8086-86a32fcc3cfb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:13.739835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc2f9-9129-4e01-8086-86a32fcc3cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:13.774325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:13.822532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-120cc2f9-9129-4e01-8086-86a32fcc3cfb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:13.823739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-120cc2f9-9129-4e01-8086-86a32fcc3cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:13.925926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 691/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:14.507980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:14.507980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:14.507980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:14.509066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 680/1371] 10.222.0.25 () {54 vars in 964 bytes} [Wed Aug 31 09:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:14.513951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d26006-d5bd-47bf-9146-fb63c43bb9bd None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:14.515229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d26006-d5bd-47bf-9146-fb63c43bb9bd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:14.515229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d26006-d5bd-47bf-9146-fb63c43bb9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:14.599540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-48d26006-d5bd-47bf-9146-fb63c43bb9bd None None] MFA Rules not processed for user `1085dcce677340bca92684d362e2fe7c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:14.600900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-48d26006-d5bd-47bf-9146-fb63c43bb9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:14.650534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:14.684608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 692/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Wed Aug 31 09:39:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:14.693094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:14.694602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:14.807110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412356d-5a02-44fc-a0ee-6b7d2dc8ce5a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:14.860503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:14.897554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:14.898103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:14.898387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:14.898682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:14.900371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:15.029224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:15.030709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe56c-a344-415a-99f6-cdb1ad826900 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:15.050860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 681/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:15.211355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:15.212787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:15.212787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:15.215343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 693/1374] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:15.220137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662818ff-64ee-44d4-8eda-ad44351a15b5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:15.220430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662818ff-64ee-44d4-8eda-ad44351a15b5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:15.220704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662818ff-64ee-44d4-8eda-ad44351a15b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:15.289985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-662818ff-64ee-44d4-8eda-ad44351a15b5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:15.291204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-662818ff-64ee-44d4-8eda-ad44351a15b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:15.313684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:15.392987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 682/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:15.684998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:15.686707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:15.838501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12eb9fe-7cc7-4074-a1c3-06a60c615d35 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:15.890019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:15.951008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:15.953246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:15.953537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:15.953814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:15.955450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:16.046011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:16.047622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde11df-df0a-4a51-be2e-86d3729d8f83 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:16.065864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 694/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:18.024298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:18.025182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:18.025620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:18.027119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 683/1377] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:18.032986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc55f-8137-4886-bf68-7ceca4f9e97d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:18.033396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc55f-8137-4886-bf68-7ceca4f9e97d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:18.033544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacc55f-8137-4886-bf68-7ceca4f9e97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:18.131799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-eaacc55f-8137-4886-bf68-7ceca4f9e97d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:18.132926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-eaacc55f-8137-4886-bf68-7ceca4f9e97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:18.140202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:18.236160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 695/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:18.535957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:18.538548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:18.716155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6908c-9032-4fcd-ba6f-2da5cbab7115 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:18.720708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:18.840846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:18.841490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:18.841773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:18.842090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:18.844268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:18.940192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:18.942042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962732-b4ba-4c1e-a757-f8c72367fa14 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:18.961786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 684/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:20.935609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:20.936258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:20.936258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:20.937738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 696/1380] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:20.943010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a51f8-6316-4b58-b572-7d22df5d5640 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:20.943517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a51f8-6316-4b58-b572-7d22df5d5640 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:20.943670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a51f8-6316-4b58-b572-7d22df5d5640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:20.955614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:21.015651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-bc1a51f8-6316-4b58-b572-7d22df5d5640 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:21.016819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bc1a51f8-6316-4b58-b572-7d22df5d5640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:21.126154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 685/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:21.430239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:21.431984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:21.574674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:21.586683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3a25a-722a-41df-aa0e-664e2c070c31 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:21.714169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:21.714777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:21.715053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:21.715331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:21.716887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:21.816228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:21.817852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0791d-f3d4-4b6a-83c1-4a9299000f2f service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:21.829490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:21.844518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 697/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:23.773605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:23.774171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:23.774346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:23.776108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 686/1383] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:23.781429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60276c-8327-437a-8f1b-9e040a774674 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:23.781851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60276c-8327-437a-8f1b-9e040a774674 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:23.781989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60276c-8327-437a-8f1b-9e040a774674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:23.859597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7d60276c-8327-437a-8f1b-9e040a774674 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:23.860901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7d60276c-8327-437a-8f1b-9e040a774674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:23.965578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 698/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:24.254901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:24.256597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:24.356543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:24.436892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf4ffb-7aaf-45ff-8b8a-17e80e280c0a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:24.572793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:24.573737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:24.574101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:24.574444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:24.577357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:24.652012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:24.700035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:24.702329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e6869-8eac-49ad-ab2f-7249606fbf56 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:24.720726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 687/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:26.672165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:26.672695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:26.672975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:26.674759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 699/1386] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:26.680148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ecf4-dabf-4e5e-a990-48d4bd298c8f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:26.680577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ecf4-dabf-4e5e-a990-48d4bd298c8f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:26.680728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ecf4-dabf-4e5e-a990-48d4bd298c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:26.749959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1550ecf4-dabf-4e5e-a990-48d4bd298c8f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:26.751186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1550ecf4-dabf-4e5e-a990-48d4bd298c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:26.852409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 688/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:27.141760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:27.143286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:27.236566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:27.323501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19ac7-cd24-469c-8d63-2233213040ba None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:27.457860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:27.458477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:27.458761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:27.459071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:27.460675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:27.498860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:27.554969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:27.557097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3619-4ab9-4d2f-a636-801407198244 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:27.579813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 700/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:29.559531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:29.560153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:29.560153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:29.561557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 689/1389] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:29.566946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54714f-269f-4147-8cb0-ee2ffa980674 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:29.567421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54714f-269f-4147-8cb0-ee2ffa980674 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:29.567629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54714f-269f-4147-8cb0-ee2ffa980674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:29.634468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ac54714f-269f-4147-8cb0-ee2ffa980674 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:29.635669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ac54714f-269f-4147-8cb0-ee2ffa980674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:29.733066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 701/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:30.019595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:30.021133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:30.052553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:30.094926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32459-3429-42e8-885f-ce99e84de273 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.095220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32459-3429-42e8-885f-ce99e84de273 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.095496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32459-3429-42e8-885f-ce99e84de273 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.096842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 702/1391] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:30.101615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e4631-8d3a-45a3-ba71-2704189a81ba None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.101899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e4631-8d3a-45a3-ba71-2704189a81ba None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.102170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e4631-8d3a-45a3-ba71-2704189a81ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.119660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:30.174980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939f9fb-c827-405e-919a-536065ae96c7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:30.178699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b44e4631-8d3a-45a3-ba71-2704189a81ba None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:30.179907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b44e4631-8d3a-45a3-ba71-2704189a81ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:30.278700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 703/1392] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:30.279683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:30.291906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:30.292511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.292787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.293096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.295192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:30.384603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:30.386233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181b0ae-c14e-4729-80ed-d4c47b5ff181 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:30.404283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 690/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:30.581231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.581772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.581772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.583126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 704/1394] 10.222.0.25 () {54 vars in 967 bytes} [Wed Aug 31 09:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:30.588572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6ad8-3ae3-48b7-8aa0-802444bd3f5c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.589210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6ad8-3ae3-48b7-8aa0-802444bd3f5c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.589395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6ad8-3ae3-48b7-8aa0-802444bd3f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.668710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cf1d6ad8-3ae3-48b7-8aa0-802444bd3f5c None None] MFA Rules not processed for user `6b91846cfa8f40109e3d3d10dec3adc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:30.670597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cf1d6ad8-3ae3-48b7-8aa0-802444bd3f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:30.724189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:30.748887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 691/1395] 10.222.0.25 () {58 vars in 1064 bytes} [Wed Aug 31 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:30.756411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:30.757970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:30.857779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:30.865529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1862025-1688-4175-843b-0ad63c49f19a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:30.939821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:30.940434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:30.940716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:30.941014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:30.942613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:31.056539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:31.058046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2066d-7a61-43d0-9544-8ac9c8585efb service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:31.077157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 705/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:31.833676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a471-1fc2-40ec-a532-48d8f24ac84b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:31.834135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a471-1fc2-40ec-a532-48d8f24ac84b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:31.834241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8a471-1fc2-40ec-a532-48d8f24ac84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:31.903979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-41b8a471-1fc2-40ec-a532-48d8f24ac84b None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:31.905218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-41b8a471-1fc2-40ec-a532-48d8f24ac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:31.991909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 692/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:32.034350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.034846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.034954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.036235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 706/1398] 10.222.0.25 () {54 vars in 943 bytes} [Wed Aug 31 09:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:32.042551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0c3a0-b38d-4ee7-ad65-c66de043ae32 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.042905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0c3a0-b38d-4ee7-ad65-c66de043ae32 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.043246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0c3a0-b38d-4ee7-ad65-c66de043ae32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.125352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f2a0c3a0-b38d-4ee7-ad65-c66de043ae32 None None] MFA Rules not processed for user `24659ce97c204823a8968c757ea40e56`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:32.126580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f2a0c3a0-b38d-4ee7-ad65-c66de043ae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.139448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:32.236332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 693/1399] 10.222.0.25 () {58 vars in 1040 bytes} [Wed Aug 31 09:39:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:32.245903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:32.247418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.272301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:32.376462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c283e5-6e46-441f-b354-3a67f02783fc None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.376748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c283e5-6e46-441f-b354-3a67f02783fc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.377048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c283e5-6e46-441f-b354-3a67f02783fc None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.378501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 694/1400] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:39:32.383214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.383494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.383769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.384789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826f307-faad-4cae-8378-897caa3f01dd None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:32.456518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:32.457838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.496777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:32.497430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.497718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.497991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.499588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.506686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:32.541646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:32.565012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 695/1401] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:32.602243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:32.603840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5995b-44c9-413c-96d6-06aa8e5b4ec3 service swift] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:32.611877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 707/1402] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:39:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:39:32.861001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:32.862224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.988378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172875d2-85a6-4c40-8527-d6c4be8dbd64 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:33.090238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:33.120549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:33.121139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:33.121416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:33.121683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:33.123306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:33.223016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:33.224591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649a0d8-f8f5-4b3b-b95d-88f38ad320d7 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:33.251585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 696/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:35.215207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:35.215768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:35.215768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:35.221331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 708/1404] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:35.222636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14390b6-c227-4680-8be4-ab9389bd88d1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:35.223059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14390b6-c227-4680-8be4-ab9389bd88d1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:35.223197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14390b6-c227-4680-8be4-ab9389bd88d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:35.302347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d14390b6-c227-4680-8be4-ab9389bd88d1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:35.303569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d14390b6-c227-4680-8be4-ab9389bd88d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:35.377144 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:35.431024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 697/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:35.722583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:35.724478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:35.897258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0b75a-c2e0-481a-acab-b975a19a72c8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:35.900247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:36.027265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:36.027788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:36.028193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:36.028541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:36.030175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:36.134321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:36.136157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ed24-a301-47eb-84e2-f09e6d437e94 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:36.159658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 709/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:38.154663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:38.155257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:38.155513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:38.156938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 698/1407] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:38.162168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b5288-85ae-48f1-a412-830e344bd12a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:38.162986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b5288-85ae-48f1-a412-830e344bd12a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:38.163455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b5288-85ae-48f1-a412-830e344bd12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:38.175345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:38.236170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-fb2b5288-85ae-48f1-a412-830e344bd12a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:38.237534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fb2b5288-85ae-48f1-a412-830e344bd12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:38.339865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 710/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:38.627392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:38.629196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:38.921273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9802493-efee-44ee-ba08-3ebcfa653ef3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:38.961380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:39.048185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:39.048872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:39.049614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:39.049891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:39.051551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:39.156078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:39.157789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549c150-3756-4c39-b028-b16042c370e5 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:39.178656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 699/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:39:48.337276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=62 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:49.174907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=82 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:54.879485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef857de-697b-4074-8b64-329e640a89b4 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:54.880066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef857de-697b-4074-8b64-329e640a89b4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:54.880066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef857de-697b-4074-8b64-329e640a89b4 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:54.881449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 711/1410] 10.222.0.25 () {54 vars in 918 bytes} [Wed Aug 31 09:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:54.886367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445724e-ff53-40e6-8a0c-c70dfdaaa55f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:54.887463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445724e-ff53-40e6-8a0c-c70dfdaaa55f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:54.887463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445724e-ff53-40e6-8a0c-c70dfdaaa55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:54.962143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4445724e-ff53-40e6-8a0c-c70dfdaaa55f None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:54.963349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4445724e-ff53-40e6-8a0c-c70dfdaaa55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:55.067767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 700/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Aug 31 09:39:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:55.083268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50ae6-3ba2-4b33-ade1-2b91bb360069 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:55.083558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50ae6-3ba2-4b33-ade1-2b91bb360069 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:55.083840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50ae6-3ba2-4b33-ade1-2b91bb360069 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:55.085210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 712/1412] 10.222.0.25 () {54 vars in 918 bytes} [Wed Aug 31 09:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:55.090591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b297f-ae8c-4d18-97ec-5bdd6792cffe None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:55.090885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b297f-ae8c-4d18-97ec-5bdd6792cffe None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:55.091164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b297f-ae8c-4d18-97ec-5bdd6792cffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:55.163990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:55.166777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-819b297f-ae8c-4d18-97ec-5bdd6792cffe None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:55.169603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-819b297f-ae8c-4d18-97ec-5bdd6792cffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:55.297463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 701/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Wed Aug 31 09:39:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:57.680831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:57.681402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:57.681402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:57.682743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:57.688775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282fdf5-f8bf-4564-8bdd-b42d291df592 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:57.689244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282fdf5-f8bf-4564-8bdd-b42d291df592 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:57.689351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282fdf5-f8bf-4564-8bdd-b42d291df592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:57.759457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a282fdf5-f8bf-4564-8bdd-b42d291df592 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:57.760673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a282fdf5-f8bf-4564-8bdd-b42d291df592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:57.814998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:57.856307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:58.150230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:58.151945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:58.306087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af66453-5bdb-4908-b16e-a3d53ce415ff None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:58.381632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:58.423384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:58.423994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:58.424274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:58.424547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:58.426167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:58.515946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:58.517579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89623a0-bbf6-4f5a-9480-96a6452bf347 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:58.535631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 714/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:59.434361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:59.434905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:59.435031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:59.436285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:59.441670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654805-a05d-4cad-87d6-67820ca336fa None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:59.442080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654805-a05d-4cad-87d6-67820ca336fa None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:59.442223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654805-a05d-4cad-87d6-67820ca336fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:59.516896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b2654805-a05d-4cad-87d6-67820ca336fa None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:59.518137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b2654805-a05d-4cad-87d6-67820ca336fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:59.545065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:59.614927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:59.762811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:59.764223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:59.907131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8920c-e4f9-4a2d-ac3d-da3c61f29f10 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:59.946979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:00.019623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:00.020159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:00.020439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:00.020712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:00.022334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:00.110878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:00.112449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a739a8-e782-4a21-ade5-1ead9c514b21 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:00.129895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 704/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:01.012656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:01.013236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:01.013236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:01.014576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:01.019383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b720-22d2-4fe5-b339-cce329f4bdbc None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:01.019784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b720-22d2-4fe5-b339-cce329f4bdbc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:01.019953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b720-22d2-4fe5-b339-cce329f4bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:01.075126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:01.099130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6e96b720-22d2-4fe5-b339-cce329f4bdbc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:01.100337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6e96b720-22d2-4fe5-b339-cce329f4bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:01.195793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:01.348736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:01.350185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:01.498988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac5385-1246-4ce0-8e89-b836d099207b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:01.511028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:01.615689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:01.616208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:01.616492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:01.616768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:01.618406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:01.708763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:01.710389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9a3de-f665-4fa3-b65d-639e5c3cb942 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:01.728569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 717/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:02.587436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:02.588001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:02.588001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:02.589890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:02.594787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4a46-4324-4d3e-b4e8-12aa12216569 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:02.595203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4a46-4324-4d3e-b4e8-12aa12216569 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:02.595321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4a46-4324-4d3e-b4e8-12aa12216569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:02.615355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:02.671089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1a5f4a46-4324-4d3e-b4e8-12aa12216569 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:02.672295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1a5f4a46-4324-4d3e-b4e8-12aa12216569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:02.766963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:02.916304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:02.917809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:03.027566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:03.063635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b1ed-6012-4fe2-a62c-bfd763d86f8f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:03.174098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:03.174628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:03.174904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:03.175173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:03.176744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:03.252455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:03.271267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:03.272863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baf7c6-7ee9-4645-b004-76da6938ffd5 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:03.292222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 707/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:04.155496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:04.156014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:04.156108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:04.157430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:04.162243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94334-c79a-437b-b219-4e5d0b7ec2a2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:04.162705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94334-c79a-437b-b219-4e5d0b7ec2a2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:04.162805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf94334-c79a-437b-b219-4e5d0b7ec2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:04.237654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-acf94334-c79a-437b-b219-4e5d0b7ec2a2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:04.238844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-acf94334-c79a-437b-b219-4e5d0b7ec2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:04.335254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:04.484605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:04.486155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:04.580579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:04.641714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c73a6-d71b-4dc2-abfd-97e8a769c8fe None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:04.754244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:04.754846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:04.755125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:04.755400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:04.757002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:04.804127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:04.847145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:04.850079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ef381-a9ce-44b5-892b-fc8f4c44fddc service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:04.869929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 720/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:05.679053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:05.679600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:05.679600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:05.680946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:05.685861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980dd90-d365-4355-a3f4-d5c4b9cb1d4e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:05.686298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980dd90-d365-4355-a3f4-d5c4b9cb1d4e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:05.686443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980dd90-d365-4355-a3f4-d5c4b9cb1d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:05.760898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f980dd90-d365-4355-a3f4-d5c4b9cb1d4e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:05.762110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f980dd90-d365-4355-a3f4-d5c4b9cb1d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:05.856846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:06.006402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:06.007824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:06.056268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:06.155506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cf5c4-59e5-4346-80c2-bba1335ddbff None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:06.268696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:06.269351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:06.269636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:06.269907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:06.271511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:06.278374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:06.364459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:06.366079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f194a-dbdb-417b-9b0f-9afbcbc7f154 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:06.384342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 710/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:07.193292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:07.193847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:07.193847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:07.195243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:07.200814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6713b83-84cf-451e-addb-0275ae9e4b79 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:07.201595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6713b83-84cf-451e-addb-0275ae9e4b79 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:07.201874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6713b83-84cf-451e-addb-0275ae9e4b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:07.277600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c6713b83-84cf-451e-addb-0275ae9e4b79 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:07.278808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c6713b83-84cf-451e-addb-0275ae9e4b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:07.347478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:07.377308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:07.527493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:07.529104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:07.546523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:07.676662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e67b8-55ec-4555-aaec-51b6f7c96259 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:07.765668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:07.791193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:07.791805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:07.792590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:07.792866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:07.794493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:07.883527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:07.885144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992e91-8cee-4288-97a7-04efb7da3443 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:07.903196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 723/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:08.708240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:08.708837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:08.708837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:08.710165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:08.715709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe73df3-9afc-49e8-8d3d-a8dd4ba3ff6d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:08.716174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe73df3-9afc-49e8-8d3d-a8dd4ba3ff6d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:08.716280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe73df3-9afc-49e8-8d3d-a8dd4ba3ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:08.790498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ffe73df3-9afc-49e8-8d3d-a8dd4ba3ff6d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:08.791700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ffe73df3-9afc-49e8-8d3d-a8dd4ba3ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:08.832559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:08.888210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:09.039641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:09.042791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:09.188201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c98831-b33a-4c7a-a7e6-2ef77cdb228b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:09.240362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:09.298356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:09.298960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:09.299239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:09.299506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:09.301124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:09.389840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:09.391424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885e312-8a8f-4da0-98af-c639aafa67c2 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:09.409316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 713/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:10.203642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.204195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.204195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.205581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:10.210704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1118b-9620-45bf-9823-1829e09feca2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.211123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1118b-9620-45bf-9823-1829e09feca2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.211264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1118b-9620-45bf-9823-1829e09feca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.286782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9cb1118b-9620-45bf-9823-1829e09feca2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:10.287994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9cb1118b-9620-45bf-9823-1829e09feca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.290053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:10.384711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:10.536941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:10.538477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.684143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20262c-6066-4ade-bdc2-51b5ce32d0da None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:10.710255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:10.798574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:10.799207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.799483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.799749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.801360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.889670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:10.891247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40ee44-49c7-4d13-9ccf-1c1cfaf08c61 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:10.909295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 726/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:12.741199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:12.741810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:12.741810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:12.743076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:12.748369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d5b8-9628-43a9-96a1-aebc78e6b2df None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:12.748797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d5b8-9628-43a9-96a1-aebc78e6b2df None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:12.748915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d5b8-9628-43a9-96a1-aebc78e6b2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:12.790001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:12.815511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4884d5b8-9628-43a9-96a1-aebc78e6b2df None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:12.816715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4884d5b8-9628-43a9-96a1-aebc78e6b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:12.913334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:13.192730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:13.194291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:13.296898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac93a71-c104-45e0-ac0a-7cdcd8f26b6e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:13.328314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:13.371618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:13.372157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:13.372439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:13.372713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:13.374369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:13.486230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:13.487666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3e96c-a670-4452-9ef1-705a217165e1 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:13.505492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 716/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:15.548445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:15.549046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:15.549046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:15.550354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:15.555445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0accc-5b96-4b03-9e81-96e24e08d8f6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:15.555871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0accc-5b96-4b03-9e81-96e24e08d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:15.556017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0accc-5b96-4b03-9e81-96e24e08d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:15.626972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-10d0accc-5b96-4b03-9e81-96e24e08d8f6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:15.628184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10d0accc-5b96-4b03-9e81-96e24e08d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:15.641576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:15.726020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:16.010186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:16.011716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:16.116483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab02d6-62f4-4859-a484-c3d305f482e2 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:16.128665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:16.189832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:16.190429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:16.190710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:16.190981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:16.192563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:16.300400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:16.301798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac9166-41cb-4b98-babc-6fafc0bc8bdc service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:16.319821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 729/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:18.540564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:18.541150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:18.541150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:18.542479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:18.548328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab5d0-ee28-4f2c-a3f2-1bd8b9c6132e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:18.548760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab5d0-ee28-4f2c-a3f2-1bd8b9c6132e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:18.548875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ab5d0-ee28-4f2c-a3f2-1bd8b9c6132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:18.598486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:18.616161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c61ab5d0-ee28-4f2c-a3f2-1bd8b9c6132e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:18.617389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c61ab5d0-ee28-4f2c-a3f2-1bd8b9c6132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:18.717663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:19.000921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:19.002486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:19.106396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178263c-7d19-4f6d-b282-9dd03eae6e48 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:19.146495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:19.180789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:19.181430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:19.181709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:19.181982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:19.183593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:19.289837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:19.291182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8501b-7e7b-4f61-b9e6-6299ee5badcf service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:19.309512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 719/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:21.336214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.336767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.336767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.338199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:21.343738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578071-7072-4691-a515-61d67de30b09 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.344167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578071-7072-4691-a515-61d67de30b09 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.344283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578071-7072-4691-a515-61d67de30b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.409772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1a578071-7072-4691-a515-61d67de30b09 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:21.410966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1a578071-7072-4691-a515-61d67de30b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.431680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:21.510646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:21.798829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:21.800476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.903164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560014f-c82e-4108-b11c-e1bb67304227 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:21.922501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:21.975781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:21.976383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.976662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.976931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.978538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:22.085368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:22.087037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd980ad-d5c1-425b-8743-2c066f16caf4 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:22.104716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 732/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:24.071375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:24.071951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:24.071951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:24.073315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:24.078728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bffdc-437c-4f34-9dc9-ea6496104df5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:24.079175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bffdc-437c-4f34-9dc9-ea6496104df5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:24.079330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bffdc-437c-4f34-9dc9-ea6496104df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:24.143222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:24.145886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-482bffdc-437c-4f34-9dc9-ea6496104df5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:24.147646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-482bffdc-437c-4f34-9dc9-ea6496104df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:24.243603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:24.533228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:24.534840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:24.638674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb0f8-6c1d-418a-981e-5e5b14b6e241 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:24.688351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:24.713325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:24.713934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:24.714217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:24.714521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:24.716118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:24.825669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:24.827009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0811-31ff-4ada-bb30-9937f894e91c service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:24.844815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 722/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:26.815467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:26.816085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:26.816085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:26.817521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:26.822448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bebe-42fd-4f88-a615-20a1e80255dc None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:26.822857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bebe-42fd-4f88-a615-20a1e80255dc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:26.822984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529bebe-42fd-4f88-a615-20a1e80255dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:26.888806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c529bebe-42fd-4f88-a615-20a1e80255dc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:26.890017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c529bebe-42fd-4f88-a615-20a1e80255dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:26.918494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:26.985167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:27.273582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:27.275116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.389673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbce6b-1301-4d4d-963c-94dbca6e0172 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:27.417578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:27.464813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:27.465446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:27.465723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:27.465990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:27.467566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.576476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:27.577951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1e503-e280-4976-b541-32f51d658f35 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:27.596093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 735/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:29.538607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:29.539160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:29.539160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:29.540506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:29.545787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efeb8-36e6-4f36-894f-70e9068f66be None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:29.546217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efeb8-36e6-4f36-894f-70e9068f66be None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:29.546338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efeb8-36e6-4f36-894f-70e9068f66be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:29.610740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f53efeb8-36e6-4f36-894f-70e9068f66be None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:29.611929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f53efeb8-36e6-4f36-894f-70e9068f66be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:29.619974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:29.708977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:29.991770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:29.993343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:30.097556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e602b-95de-4fa2-bdf8-182f0322db6a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:30.152319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:30.171717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:30.172318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:30.172597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:30.172868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:30.174493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:30.289741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:30.291087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd3c9d-bdbb-4152-a3e8-10daddd2ecdd service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:30.311121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 725/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:32.266607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:32.267135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:32.267261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:32.268495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:32.273810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef126-c82d-41ab-8e05-fc3a0d54d22b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:32.274257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef126-c82d-41ab-8e05-fc3a0d54d22b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:32.274368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef126-c82d-41ab-8e05-fc3a0d54d22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:32.342920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6abef126-c82d-41ab-8e05-fc3a0d54d22b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:32.344039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6abef126-c82d-41ab-8e05-fc3a0d54d22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:32.388475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:32.450066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:32.739046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:32.740583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:32.844890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9676ded-bf2f-4bb4-b017-9ec8ec319bb4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:32.882751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:32.918066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:32.918674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:32.918956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:32.919226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:32.920815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:33.027990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:33.029389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7753b21-70c3-4fe2-8a4e-e24cd1ee09ec service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:33.046957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 738/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:34.961999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.962573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.962573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.963890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:34.969102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beebe9d-d970-415f-a8d7-de8a6607f58d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.969488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beebe9d-d970-415f-a8d7-de8a6607f58d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.969671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beebe9d-d970-415f-a8d7-de8a6607f58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:35.035096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1beebe9d-d970-415f-a8d7-de8a6607f58d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:35.036303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1beebe9d-d970-415f-a8d7-de8a6607f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:35.053939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:35.133388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:35.412097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:35.413699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:35.518646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af08f-7251-4624-9b30-1d73cde662e6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:35.585129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:35.592717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:35.593357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:35.593635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:35.593905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:35.595487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:35.704352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:35.706766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b16bd-244c-4346-a148-402f27bf0bac service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:35.725336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 728/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:37.701442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:37.701958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:37.702057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:37.703302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:37.708411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f379004-5a5e-47ee-97f8-518d93eef75b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:37.708885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f379004-5a5e-47ee-97f8-518d93eef75b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:37.709036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f379004-5a5e-47ee-97f8-518d93eef75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:37.780098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0f379004-5a5e-47ee-97f8-518d93eef75b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:37.781342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0f379004-5a5e-47ee-97f8-518d93eef75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.829929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:37.880073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:38.162679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:38.164191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:38.267650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776b71-5182-435c-988e-9b108e2219e7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:38.314589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:38.340978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:38.341528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:38.341807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:38.342079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:38.343664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:38.451811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:38.453273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc399966-d95a-4469-ac54-a868d433941b service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:38.471413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 741/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:47.878274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=22 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:48.469196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=66 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:58.164380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97405a6-0378-4b88-92b8-c592c830fee6 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.164922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97405a6-0378-4b88-92b8-c592c830fee6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.164922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97405a6-0378-4b88-92b8-c592c830fee6 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.166334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 730/1471] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:58.172485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55a709-b244-4771-a36f-227939edb52f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.172918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55a709-b244-4771-a36f-227939edb52f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.173111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55a709-b244-4771-a36f-227939edb52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.238197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-aa55a709-b244-4771-a36f-227939edb52f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:58.239399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-aa55a709-b244-4771-a36f-227939edb52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.338382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 742/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:58.635530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.636327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.636806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.638566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 731/1473] 10.222.0.25 () {54 vars in 967 bytes} [Wed Aug 31 09:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:58.643689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62d0e-b22e-4e14-8a51-74dd6af03f17 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.643978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62d0e-b22e-4e14-8a51-74dd6af03f17 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.644254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e62d0e-b22e-4e14-8a51-74dd6af03f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.722029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-57e62d0e-b22e-4e14-8a51-74dd6af03f17 None None] MFA Rules not processed for user `6b91846cfa8f40109e3d3d10dec3adc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:58.723340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-57e62d0e-b22e-4e14-8a51-74dd6af03f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.745348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:58.801310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 743/1474] 10.222.0.25 () {58 vars in 1064 bytes} [Wed Aug 31 09:40:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:58.809460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:58.811949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.914234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9abac-af99-4238-8394-be7a103a8b16 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:58.985690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:58.986293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.986575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.986851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.988466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:59.042053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:59.098244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:59.099671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9202504-fc67-4e90-91ba-6b6b45bde9e5 service glance] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:59.117673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 732/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:01.097048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:01.097533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:01.097637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:01.098983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 744/1476] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:01.104096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be277c-9a89-4c30-b20a-ed7c3eae7961 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:01.104568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be277c-9a89-4c30-b20a-ed7c3eae7961 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:01.104677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be277c-9a89-4c30-b20a-ed7c3eae7961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:01.169739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-46be277c-9a89-4c30-b20a-ed7c3eae7961 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:01.170939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-46be277c-9a89-4c30-b20a-ed7c3eae7961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:01.272416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 733/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:01.544043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:01.545684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:01.670378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18889b4b-248b-412c-a684-a3f3ee2c5793 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:01.719573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:01.764354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:01.764891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:01.765203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:01.765480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:01.766471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:01.768289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:41:01.768289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:41:01.768289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:41:01.769481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3eb5d-906f-41ac-aab4-db7c30db0559 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:01.792528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 745/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Wed Aug 31 09:41:01 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:41:03.703295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:03.703868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:03.703868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:03.705239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 734/1479] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:03.710701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f2778-8f28-4cd5-9735-fcc0e33c09ce None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:03.711139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f2778-8f28-4cd5-9735-fcc0e33c09ce None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:03.711250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f2778-8f28-4cd5-9735-fcc0e33c09ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:03.779038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-979f2778-8f28-4cd5-9735-fcc0e33c09ce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:03.780163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-979f2778-8f28-4cd5-9735-fcc0e33c09ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:03.874397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 746/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:04.165575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:04.167253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:04.177065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:04.314360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f1c80-7497-47b1-b523-81fd2c160517 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:04.393367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:04.425865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:04.426466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:04.426748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:04.427021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:04.428607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:04.515821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:04.517409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1d87-ccf2-41b2-8aa8-0c32df80ab4a service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:04.535013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 735/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:06.629604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:06.630142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:06.630279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:06.631546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 747/1482] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:06.637704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae442-6e60-44f5-9c81-e600c21543a3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:06.638118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae442-6e60-44f5-9c81-e600c21543a3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:06.638255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae442-6e60-44f5-9c81-e600c21543a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:06.704184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-339ae442-6e60-44f5-9c81-e600c21543a3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:06.705443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-339ae442-6e60-44f5-9c81-e600c21543a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:06.752028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:06.803034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 736/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:07.092500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:07.094141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:07.117995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:07.197058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521bbb-26a8-4fe8-b5be-43b00a4a0bf6 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:07.269688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:07.270296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:07.270574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:07.270844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:07.272435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:07.347848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:07.382255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:07.383688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d850bd4-9493-40c8-a338-e58d90285b44 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:07.401336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 748/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:09.344099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:09.344692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:09.344811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:09.346311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 737/1485] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:09.351556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1606e-7497-489f-a85b-7ca5ec82212d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:09.352010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1606e-7497-489f-a85b-7ca5ec82212d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:09.352119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1606e-7497-489f-a85b-7ca5ec82212d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:09.417110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-93c1606e-7497-489f-a85b-7ca5ec82212d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:09.418804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-93c1606e-7497-489f-a85b-7ca5ec82212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:09.513831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 749/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:09.808125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:09.809729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:09.913147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c6a6e-71dd-4cea-a925-31b70e877190 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:09.983200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:09.985126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:09.985776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:09.986268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:09.986704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:09.988484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:10.097140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:10.098569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbfcd3-7ca4-49e5-b34a-011ad09eb242 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:10.116435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 738/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:12.222470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.223028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.223028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.224358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 750/1488] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:12.230186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0266-b43b-4042-ba05-d062b7b6e81d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.230599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0266-b43b-4042-ba05-d062b7b6e81d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.230742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0266-b43b-4042-ba05-d062b7b6e81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.296793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-06ea0266-b43b-4042-ba05-d062b7b6e81d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:12.298026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-06ea0266-b43b-4042-ba05-d062b7b6e81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.356683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:12.401041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 739/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:12.691994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:12.694543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.733201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:12.800461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a83bde-0aa8-46ed-8e15-e9bbd82f4dc1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:12.871956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:12.872552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:12.872833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:12.873133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:12.874710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:12.953887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:12.982901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:12.984333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d662-6212-4111-bdde-f4e5876d4ce9 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:13.003510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 751/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:15.111307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:15.111753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:15.111876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:15.113244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 740/1491] 10.222.0.25 () {54 vars in 927 bytes} [Wed Aug 31 09:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:15.118382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6c242-2957-4290-9145-47e5e39f4812 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:15.118802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6c242-2957-4290-9145-47e5e39f4812 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:15.118930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6c242-2957-4290-9145-47e5e39f4812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:15.183327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-44b6c242-2957-4290-9145-47e5e39f4812 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:15.184514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-44b6c242-2957-4290-9145-47e5e39f4812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:15.279143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 752/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:15.573337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:15.574914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:15.677929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3c98e-8f7d-4034-8ec8-8f2233df4e73 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:15.754458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:15.754988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:15.755272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:15.755548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:15.757209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:15.766834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:15.865764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:15.867185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754324fb-ba49-48dc-ac41-b13836fb8f0d service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:15.884692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 741/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:25.277279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=82 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.882456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=50 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:28.359321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:28.359883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:28.359883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:28.435456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:28.436650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:28.532561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 753/1494] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:28.540498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:28.541049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:28.541270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:28.617206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:28.618336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:28.713981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 742/1495] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:28.718650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:28.719852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:28.731168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e3da7-873b-4029-adcb-3789503cd5d3 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:28.793813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:28.829899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:28.830507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:28.830792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:28.831067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:28.831703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:42:28.832030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:28.833141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:28.833141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:42:28.833141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:42:28.834024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:28.845075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 754/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:42:28.853411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:28.862576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7d91-8392-4ca2-9967-ddd13fd00e98 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:28.923527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:28.958120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:28.958722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:28.959002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:28.959274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:28.959892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:28.961894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:28.961894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:42:28.961894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:42:28.963084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:29.019579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 743/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:42:28 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:42:29.023812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:29.025352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.059062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb04e-32d2-4b16-bc16-261ae276f8a5 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:29.154916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:29.155522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.155799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.156070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.156723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:29.157781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:29.157781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:42:29.157781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:42:29.158998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:29.200658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 755/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:42:29 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:42:29.205043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:29.206630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.268871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7aec7-98da-471e-a21a-abbae7464a66 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:29.352927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:29.364701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:29.365329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.365607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.365875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.366509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:29.367547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:29.367547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:42:29.367547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:42:29.368436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:29.381771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 744/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:42:29.385955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:29.387491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.398772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ffadd-cddd-467c-879c-7571a9d7d582 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:29.424612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:29.505520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:29.506055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.506337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.506610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.507245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:29.508256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:29.508256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:42:29.508256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:42:29.509164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48895caf-e0fc-46f1-8535-b64d5813743b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:29.521380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 756/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:42:29.526429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:29.528027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.561556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bfe79-3408-4656-883c-7bf2d03be0a5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:29.661015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:29.661676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.661957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.662234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] PATH_INFO: `/v3/projects/67532d9a4fb34d4489aa72d935e680af/users/b4eac2b098a24f6ab551b8e4d87dc052/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.684154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67532d9a4fb34d4489aa72d935e680af, user_id=b4eac2b098a24f6ab551b8e4d87dc052, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:29.685194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:29.685194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:42:29.685194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:42:29.687129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7752826-926d-41fc-9fff-8f665ee8ea59 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:29.711786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 745/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:42:29 2022] PUT /identity/v3/projects/67532d9a4fb34d4489aa72d935e680af/users/b4eac2b098a24f6ab551b8e4d87dc052/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:42:29.724445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e7e8-7cb4-431b-8de1-80ed1dc43f04 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.724735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e7e8-7cb4-431b-8de1-80ed1dc43f04 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.725066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02e7e8-7cb4-431b-8de1-80ed1dc43f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.781798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-bf02e7e8-7cb4-431b-8de1-80ed1dc43f04 None None] MFA Rules not processed for user `b4eac2b098a24f6ab551b8e4d87dc052`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:29.782928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bf02e7e8-7cb4-431b-8de1-80ed1dc43f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:29.860067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 757/1502] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:29.873030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.873328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.873611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.874941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 746/1503] 10.222.0.25 () {54 vars in 964 bytes} [Wed Aug 31 09:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:29.879564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc606e-7543-4241-aab8-8a9f6f1a49fa None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:29.879860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc606e-7543-4241-aab8-8a9f6f1a49fa None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:29.880141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc606e-7543-4241-aab8-8a9f6f1a49fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:29.910878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:29.959247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e8dc606e-7543-4241-aab8-8a9f6f1a49fa None None] MFA Rules not processed for user `1085dcce677340bca92684d362e2fe7c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:29.960469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e8dc606e-7543-4241-aab8-8a9f6f1a49fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.035651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 758/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Wed Aug 31 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:30.045202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:30.046718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.097371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.153780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e299e8-005c-4c8c-820f-dd4863e51755 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:30.226176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:30.226786 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:30.227070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:30.227348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:30.228947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.333758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:30.335143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:30.338870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.354734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 747/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:30.400372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5979ee8-f70d-4f17-954c-c5e61a9089ad None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:30.400840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5979ee8-f70d-4f17-954c-c5e61a9089ad None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:30.400987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5979ee8-f70d-4f17-954c-c5e61a9089ad None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:30.402200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 759/1506] 10.222.0.25 () {52 vars in 830 bytes} [Wed Aug 31 09:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:42:30.405796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:30.407013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.505954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b572ec-4b44-4807-86df-bccc03231d8e service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:30.602837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:30.604551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:30.604731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:30.605039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] PATH_INFO: `/v3/users/b4eac2b098a24f6ab551b8e4d87dc052` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:30.620321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4eac2b098a24f6ab551b8e4d87dc052)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:30.621360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:30.621360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:42:30.621360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:42:30.622567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:30.632111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.690003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4eac2b098a24f6ab551b8e4d87dc052'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:30.699117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4eac2b098a24f6ab551b8e4d87dc052'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:30.722540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4eac2b098a24f6ab551b8e4d87dc052'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:30.737593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4eac2b098a24f6ab551b8e4d87dc052'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:30.738171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4eac2b098a24f6ab551b8e4d87dc052'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:30.743895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 748/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:42:30 2022] DELETE /identity/v3/users/b4eac2b098a24f6ab551b8e4d87dc052 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:42:30.748927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:30.749412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:30.749912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:30.805920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:30.807736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:30.883204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:30.896941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 760/1508] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:30.909094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:30.910374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:31.011756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfac24-a719-4691-85c4-4824d51c5927 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:31.065858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:31.088522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:31.088971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:31.089329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:31.089600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:31.091251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:31.199537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:31.201214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334bc61-5e96-4bd6-a96f-8751fd92fb45 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:31.223488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 749/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:31.593737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:31.595078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:31.607366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdedb4c-4c36-4bf1-90e6-cc508f63e53b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:31.706637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:31.707270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:31.707554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:31.707828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] PATH_INFO: `/v3/projects/67532d9a4fb34d4489aa72d935e680af` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:31.718331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67532d9a4fb34d4489aa72d935e680af)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:31.719305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:31.719305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:42:31.719305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:42:31.720499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:31.778211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67532d9a4fb34d4489aa72d935e680af', 'request_id': 'req-caf11996-e450-4e08-933f-e074572c6b1f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:31.784887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67532d9a4fb34d4489aa72d935e680af', 'request_id': 'req-caf11996-e450-4e08-933f-e074572c6b1f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:31.799704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67532d9a4fb34d4489aa72d935e680af', 'request_id': 'req-caf11996-e450-4e08-933f-e074572c6b1f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:31.828568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] The token cache is being invalidate because project 67532d9a4fb34d4489aa72d935e680af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:42:31.828859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:31.829320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-caf11996-e450-4e08-933f-e074572c6b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:42:31.830698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 761/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:42:31 2022] DELETE /identity/v3/projects/67532d9a4fb34d4489aa72d935e680af => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:42:41.219461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=62 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.826425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:11.770016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:11.770706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:11.770706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:11.772688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:11.778421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f66c5-9f4d-44d4-9364-b04e7f5d60f5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:11.778963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f66c5-9f4d-44d4-9364-b04e7f5d60f5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:11.778963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f66c5-9f4d-44d4-9364-b04e7f5d60f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:11.862090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ea1f66c5-9f4d-44d4-9364-b04e7f5d60f5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:11.863304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ea1f66c5-9f4d-44d4-9364-b04e7f5d60f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:11.964530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:12.119743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:12.121860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:12.239038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7801049-1c48-4444-aeca-4427b570b4bf None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:12.245083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:12.245695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:12.245978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:12.246256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:12.254311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:12.256464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef89df3-7334-427c-95fa-64ad8e737d45 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:12.265827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 751/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:13.093505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad79d-20a5-4c75-b4c4-09700b8b1f7c None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:13.093964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad79d-20a5-4c75-b4c4-09700b8b1f7c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:13.094432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad79d-20a5-4c75-b4c4-09700b8b1f7c None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:13.096786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:44:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:13.103448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd1416-2aa4-417f-9dc9-caf1fe799b64 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:13.103903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd1416-2aa4-417f-9dc9-caf1fe799b64 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:13.104022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd1416-2aa4-417f-9dc9-caf1fe799b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:13.124872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-fdbd1416-2aa4-417f-9dc9-caf1fe799b64 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:13.126079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fdbd1416-2aa4-417f-9dc9-caf1fe799b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:13.133938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:13.284976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdad79d-20a5-4c75-b4c4-09700b8b1f7c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:13.299577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdad79d-20a5-4c75-b4c4-09700b8b1f7c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:13.305588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:13.306183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:13.306462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:13.306731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:13.314661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:13.316292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fba74f-cacc-4b55-b34b-530f232fe87c service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:13.325755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 764/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:14.099477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4872e-dc92-4f00-baf2-20b8e98db724 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:14.099772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4872e-dc92-4f00-baf2-20b8e98db724 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:14.100053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4872e-dc92-4f00-baf2-20b8e98db724 None None] PATH_INFO: `/v3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:14.101364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 753/1517] 10.222.0.72 () {54 vars in 925 bytes} [Wed Aug 31 09:44:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:44:14.108750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ceb5-a30c-41ad-8d36-f619016f7ebf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:14.109277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ceb5-a30c-41ad-8d36-f619016f7ebf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:14.109387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ceb5-a30c-41ad-8d36-f619016f7ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:14.168214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6491ceb5-a30c-41ad-8d36-f619016f7ebf None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:14.169468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6491ceb5-a30c-41ad-8d36-f619016f7ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:14.186520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:14.240540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 765/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Aug 31 09:44:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:44:23.120034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=56 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:23.532152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9405-832a-4c80-b892-c7931a89a53c None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:23.532433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9405-832a-4c80-b892-c7931a89a53c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:23.532713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9405-832a-4c80-b892-c7931a89a53c None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:23.534100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 754/1519] 10.222.0.25 () {54 vars in 961 bytes} [Wed Aug 31 09:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:23.539411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd13e7-489b-48ee-99ca-50297616747b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:23.539888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd13e7-489b-48ee-99ca-50297616747b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:23.540063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd13e7-489b-48ee-99ca-50297616747b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:23.561058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5acd13e7-489b-48ee-99ca-50297616747b None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:23.562264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5acd13e7-489b-48ee-99ca-50297616747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:23.570934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 766/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Aug 31 09:44:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:23.578653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9405-832a-4c80-b892-c7931a89a53c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:23.592000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9405-832a-4c80-b892-c7931a89a53c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:23.596843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:23.597461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:23.597738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:23.598007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:23.606715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:23.608288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e3a25-f62c-4959-b9b1-b65e2072935e service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:23.618581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 755/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:24.549650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf30ee-75b3-468a-a6d3-dbf735c842da None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:24.550183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf30ee-75b3-468a-a6d3-dbf735c842da None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:24.550183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf30ee-75b3-468a-a6d3-dbf735c842da None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:24.551560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:24.556362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7476-516b-430d-9f0f-c4f999508969 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:24.556766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7476-516b-430d-9f0f-c4f999508969 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:24.556906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7476-516b-430d-9f0f-c4f999508969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:24.576065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dfbe7476-516b-430d-9f0f-c4f999508969 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:24.577263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dfbe7476-516b-430d-9f0f-c4f999508969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:24.585455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:24.738822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf30ee-75b3-468a-a6d3-dbf735c842da None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:24.754319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf30ee-75b3-468a-a6d3-dbf735c842da None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:24.760288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:24.760892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:24.761208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:24.761486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:24.769724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:24.771349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764c9e4-0759-4863-a3ea-2faf3a203080 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:24.781047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 768/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:25.596586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d894-9ad3-4470-bd93-6ebc77f2d80f None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:25.597017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d894-9ad3-4470-bd93-6ebc77f2d80f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:25.597190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d894-9ad3-4470-bd93-6ebc77f2d80f None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:25.598550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:25.603797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487f6c-81de-4a3d-a893-86f2a17dcac0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:25.604194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487f6c-81de-4a3d-a893-86f2a17dcac0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:25.604351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487f6c-81de-4a3d-a893-86f2a17dcac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:25.624192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-48487f6c-81de-4a3d-a893-86f2a17dcac0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:25.625402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-48487f6c-81de-4a3d-a893-86f2a17dcac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:25.633255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:25.781789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d894-9ad3-4470-bd93-6ebc77f2d80f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:25.796543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d894-9ad3-4470-bd93-6ebc77f2d80f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:25.801970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:25.802554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:25.802831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:25.803155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:25.812595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:25.814220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae570e-df05-4ad9-8c8c-b60677702cff service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:25.824038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 758/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:29.854896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:29.855460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:29.855460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:29.856756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:29.862026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852a83e-1171-4bdb-90ad-30f6c7e58dce None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:29.862434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852a83e-1171-4bdb-90ad-30f6c7e58dce None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:29.862566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852a83e-1171-4bdb-90ad-30f6c7e58dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:29.883077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c852a83e-1171-4bdb-90ad-30f6c7e58dce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:29.884260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c852a83e-1171-4bdb-90ad-30f6c7e58dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:29.892081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:30.178776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:30.180452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:30.257974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947c839-d518-4d77-872b-1fe6ef4a6e74 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:30.261508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:30.262111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:30.262392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:30.262667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:30.271351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:30.272838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386286ef-d2f4-4d4c-a93e-34240b75c13d service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:30.281973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 771/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:32.181383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:32.181900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:32.182020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:32.183271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:32.188485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ea6b-f803-44e2-9fd8-4052cf8cd998 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:32.188909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ea6b-f803-44e2-9fd8-4052cf8cd998 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:32.189268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ea6b-f803-44e2-9fd8-4052cf8cd998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:32.210694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c0b5ea6b-f803-44e2-9fd8-4052cf8cd998 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:32.211901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c0b5ea6b-f803-44e2-9fd8-4052cf8cd998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:32.219649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:32.505911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:32.507605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:32.520422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36d105-6b3d-4952-ab35-37083b5ae47b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:32.523918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:32.524513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:32.524792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:32.525103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:32.533757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:32.535151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965cb65-05e0-456f-9017-c556e0dfca30 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:32.544846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 761/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:36.440840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:36.441440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:36.441440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:36.442743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:36.448983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d50e-0391-41ec-bf82-d58c1b319b50 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:36.449416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d50e-0391-41ec-bf82-d58c1b319b50 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:36.449545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3d50e-0391-41ec-bf82-d58c1b319b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:36.468388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a5a3d50e-0391-41ec-bf82-d58c1b319b50 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:36.469602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a5a3d50e-0391-41ec-bf82-d58c1b319b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:36.477189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:37.048572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:37.050291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:37.234036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:37.248067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927b220-88e0-4810-84f1-eec778637d62 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:37.251566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:37.252189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:37.252468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:37.252739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:37.262370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:37.263803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c35f64-3f72-44cd-a5df-c951ee13a5a7 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:37.273147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 774/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:39.207057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:39.207612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:39.207612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:39.209502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:39.214803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28fed-4cee-4d23-9ed3-52b36b8ec845 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:39.215234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28fed-4cee-4d23-9ed3-52b36b8ec845 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:39.215340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28fed-4cee-4d23-9ed3-52b36b8ec845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:39.234180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8db28fed-4cee-4d23-9ed3-52b36b8ec845 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:39.235361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8db28fed-4cee-4d23-9ed3-52b36b8ec845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:39.247341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:44:39.807706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:39.809406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:39.822520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9730d6-c6a5-4dba-9de5-f40b5c2bef72 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:39.826103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:39.826696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:39.826979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:39.827264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:39.836181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:39.837635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e82086-d5c1-4a5f-b5be-6c7b9383db6e service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:39.846887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 764/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:49.228930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=11 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:49.842839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=33 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:10.876847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:10.877573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:10.877573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:10.899350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:10.900547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:10.911907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 776/1540] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:10.921110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:10.921674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:10.921674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:10.950022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:10.951730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:10.962905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 765/1541] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:10.967782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:10.979798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53e28c-b7cc-43ea-a050-381260292316 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:10.988781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:10.989441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:10.989751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:10.989990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:10.990638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:10.990937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:10.992535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.007436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 777/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:11.016222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.026821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63181421-fabc-40d5-9dee-62c3e1e83e12 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.037993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.038893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.039406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.039898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.040743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.043066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.105418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 766/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/projects => generated 379 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:11.109899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.126789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5320cf9-3d0a-410f-a03b-ea55428b3626 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.209287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.209929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.210213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.210485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.211135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.213430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.256806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 778/1544] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:11.261323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.276170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5db8-1c00-47b0-ab44-5381a8bd4c1b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.289083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.290085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.290577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.291089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.292076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.294044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.311668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 767/1545] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:11.316058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.331648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb2142-a38c-400b-bd4a-c8aebe92cfe5 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.338498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.339267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.339553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.339828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.340466 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.342221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660b49-6386-4e34-96c4-8e916cd496c9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.356113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 779/1546] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:11.360505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.373660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0d92-1a52-4558-8c5f-9526b847cec3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.381880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.382968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.383488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.383984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] PATH_INFO: `/v3/projects/3f3531fb48774df8a26ee722f3eb837e/users/5b9f29fab9e24e98a632416f2160182c/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.428230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3531fb48774df8a26ee722f3eb837e, user_id=5b9f29fab9e24e98a632416f2160182c, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.431559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.461411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 768/1547] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:11 2022] PUT /identity/v3/projects/3f3531fb48774df8a26ee722f3eb837e/users/5b9f29fab9e24e98a632416f2160182c/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:11.466908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16f30-540b-4efb-a0b0-4168b36a2d6d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.467376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16f30-540b-4efb-a0b0-4168b36a2d6d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.467662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16f30-540b-4efb-a0b0-4168b36a2d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.508944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4da16f30-540b-4efb-a0b0-4168b36a2d6d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.510230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4da16f30-540b-4efb-a0b0-4168b36a2d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.606389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 780/1548] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.617190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.630885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60552537-ae0d-4514-9c63-f0e866cd199f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.631228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60552537-ae0d-4514-9c63-f0e866cd199f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.631439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60552537-ae0d-4514-9c63-f0e866cd199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.668488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-60552537-ae0d-4514-9c63-f0e866cd199f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.669749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-60552537-ae0d-4514-9c63-f0e866cd199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.691672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 781/1549] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:11.700453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234c30-93fc-4931-9482-57041616cee9 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.700735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234c30-93fc-4931-9482-57041616cee9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.701305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234c30-93fc-4931-9482-57041616cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.727178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3ed8d-95b5-430b-a678-0f40c0ded0a7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.732389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-15234c30-93fc-4931-9482-57041616cee9 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.734782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-15234c30-93fc-4931-9482-57041616cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.738133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.739018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.740338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.741319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.755471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 782/1550] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.758444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40533-2e3d-488d-b8da-9659da9dcdf0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.758970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40533-2e3d-488d-b8da-9659da9dcdf0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.759479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40533-2e3d-488d-b8da-9659da9dcdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.764545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.766815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.814325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 769/1551] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:11.816111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.818266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-91e40533-2e3d-488d-b8da-9659da9dcdf0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.825171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-91e40533-2e3d-488d-b8da-9659da9dcdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.840273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0d2a8-30bf-4a63-aba7-fc74461ee914 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.851477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 783/1552] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.853168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a185-0359-4f22-b3e3-c1fb7fef30ce None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.853624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a185-0359-4f22-b3e3-c1fb7fef30ce None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.853982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a185-0359-4f22-b3e3-c1fb7fef30ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.863982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.864614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.865101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.865516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.866180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:11.866488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.867846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:11.867846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:46:11.867846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:46:11.868787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614752ba-257a-4884-b592-c69a39e6f301 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.892983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d610a185-0359-4f22-b3e3-c1fb7fef30ce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.894933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d610a185-0359-4f22-b3e3-c1fb7fef30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.896064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 770/1553] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:11.900211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.900776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.901099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.905247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 784/1554] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.907038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd313b0-ffe4-45a3-935e-438bc6f8b606 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.907516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd313b0-ffe4-45a3-935e-438bc6f8b606 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.908207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd313b0-ffe4-45a3-935e-438bc6f8b606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.934184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-fdd313b0-ffe4-45a3-935e-438bc6f8b606 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.935775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fdd313b0-ffe4-45a3-935e-438bc6f8b606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.938265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.939453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.947902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 785/1555] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.949582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aa72-7d05-4c03-a676-05dfc1ea17ce None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.950071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aa72-7d05-4c03-a676-05dfc1ea17ce None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.950522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aa72-7d05-4c03-a676-05dfc1ea17ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.953324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 771/1556] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.956574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.976194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc948c-7814-4a37-8c50-7d1f121413a3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.976398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-00a3aa72-7d05-4c03-a676-05dfc1ea17ce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:11.977512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-00a3aa72-7d05-4c03-a676-05dfc1ea17ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.982312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.983174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.983465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.983774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.984669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.987711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41756cba-4325-4162-8563-15a88906d5cc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.990851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 786/1557] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.991962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2c45-2e01-4b74-afda-83d5e846479e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.992251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2c45-2e01-4b74-afda-83d5e846479e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.992735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2c45-2e01-4b74-afda-83d5e846479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.012144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-eafe2c45-2e01-4b74-afda-83d5e846479e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.013525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-eafe2c45-2e01-4b74-afda-83d5e846479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.020045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 772/1558] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.021742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 787/1559] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.023085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71cd3-2603-49a3-b568-8dd9b84f37a4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.023800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71cd3-2603-49a3-b568-8dd9b84f37a4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.023928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71cd3-2603-49a3-b568-8dd9b84f37a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.024634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2bf03-4991-4153-a883-9b001eaec837 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.024929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2bf03-4991-4153-a883-9b001eaec837 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.025402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2bf03-4991-4153-a883-9b001eaec837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.047485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-bee71cd3-2603-49a3-b568-8dd9b84f37a4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.048706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bee71cd3-2603-49a3-b568-8dd9b84f37a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.050586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6ac2bf03-4991-4153-a883-9b001eaec837 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.052608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6ac2bf03-4991-4153-a883-9b001eaec837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.059061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:12.123541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:12.130556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 788/1560] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.131892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.132453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.132592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.147711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 773/1561] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.149484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.149872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.150237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.152330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.153588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.162975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 789/1562] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.163784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.173440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a7a2b-a4b6-4995-a04c-dfca1fa29d99 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.177792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.179205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.179422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.180163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.180524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.180731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.182160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.182160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.183638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.190839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 774/1563] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.192114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.196822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 790/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.198401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.203332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f8ef6-6fb0-46a2-909a-47c4b2520d04 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.208739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.209405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.209681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.209951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.210580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.211530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fedb4d-0ec8-4001-a7c9-f795067c1a99 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.211654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:12.211654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:46:12.211654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:46:12.212800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.216212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.216882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.217258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.217599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.218342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.218711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.220543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930a7db-52ac-49ed-8a41-647e343be75a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.241955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 775/1565] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/users => generated 291 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.241955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.252079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 791/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.254353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381995-a713-4f74-9850-bab2592451fb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.254683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381995-a713-4f74-9850-bab2592451fb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.255009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c381995-a713-4f74-9850-bab2592451fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.256901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205782-e46e-403b-9bdd-6d2d177b11c4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.262871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.263609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.263995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.264579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.266207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.266613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.268927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.278076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1c381995-a713-4f74-9850-bab2592451fb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.279879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1c381995-a713-4f74-9850-bab2592451fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.284857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 776/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.292096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.297225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 792/1568] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.298057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c381995-a713-4f74-9850-bab2592451fb None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.303841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61631241-db8b-49bb-bdc7-c26ef95f985e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.308605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c381995-a713-4f74-9850-bab2592451fb None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.311352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.313478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.314037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.314380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.315289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.315680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.316031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.316373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.316775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.317171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.317698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.318808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:12.318808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:46:12.318808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:46:12.319769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.321450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b804e3-1bfa-4e86-b9c5-798a00460902 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.337150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 777/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.338979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.339458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.339908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.359224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 793/1570] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.360261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.367830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:12.369445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:12.430904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c754f1e-8d5e-43ce-b1c2-062f3f690e31 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.435806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.436386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.436671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.436937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.437398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 778/1571] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:12.438447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.441958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.443868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.450453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e60d-1e32-43ff-859e-9aac6866aaa0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.454775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 794/1572] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:12.454775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.458659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.459342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.459616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.459889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.460520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.462489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.467875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63654b56-4797-4fd9-9e74-ffcb4b209099 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.472779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.473163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.473438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.473723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.474379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.476439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.495736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 779/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.496808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.512857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 795/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.513605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.545871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:12.579313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd60e5-eaf5-4829-b55d-05d10e15d79b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.593245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5765-8497-4c72-87b0-4bdc0d429ec9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.599173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.599915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.600201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.600475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.601260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.603188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.617476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:12.649646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 796/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.650753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.673602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.674758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.675041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.675318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.675969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.676281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.679097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.694221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 780/1576] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.695459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.706054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86d4f6-2122-4335-9bff-aa1bde1228a8 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.744168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea25b3-103d-4e9e-b580-81ce773a4a69 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.750456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.751324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.751778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.752763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.753601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.756308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.780774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.781471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.781812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.782093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.782745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:12.783161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.786191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.788875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 797/1577] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.791205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.803571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 781/1578] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:12.804772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.805383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94193a62-e20d-4a98-b7bb-58d26bdb89ab admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.810397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.811515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.811954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.812407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.813251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.816808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.819714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211870-e98d-4331-bea5-67ef59f9a831 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.828164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.828918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.829237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.829517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.830044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 798/1579] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:12.830677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.831231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.832875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.841702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca057b-2b14-414c-84f7-bddbc02d83aa admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.852068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.853477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.854486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.854944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.855742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.858639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.881222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 782/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/users => generated 302 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.882195 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.892490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 799/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.893895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.894478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42ffa7-616a-44d8-b879-25990d2c292a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.900164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.901326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.901326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.901504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.902092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.904249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.910756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cd937-976f-4ca1-9f29-7a518247be75 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.919172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.920339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.920339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.920547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.921990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.924815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.931523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 783/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.935832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.948176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e49f1-67a8-4494-81f9-37efc3a44865 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:12.954487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:12.955487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:12.955837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:12.956047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:12.957429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:12.961086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 800/1583] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:12.961086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.961452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:12.976901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 784/1584] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:12.976901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:12.990866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57ef6c-69b2-4725-88ad-354c993e61b9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.027676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:13.047746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb40c9-e59e-4300-951e-3d2b8694b0da admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.053444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.054071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.054405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.054685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.055323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.057530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.078055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.079137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.079621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.080026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] PATH_INFO: `/v3/projects/0db155f32ed94921a444706298b1f2d9/users/334506441b284e4c8472a391622da7a6/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.092112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 801/1585] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:12 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:13.094350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:13.095268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.101238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db155f32ed94921a444706298b1f2d9, user_id=334506441b284e4c8472a391622da7a6, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.105172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.126322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 785/1586] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:12 2022] PUT /identity/v3/projects/0db155f32ed94921a444706298b1f2d9/users/334506441b284e4c8472a391622da7a6/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:13.127569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.183079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d3cbe-be69-4337-80eb-a0aca0b898ca admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.223879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b3c45-7fe3-4f6f-972e-be7798661391 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.229969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.231211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.231504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.231780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.232759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.234574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.250676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 786/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.251686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.263083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79fafc-a327-4573-b8f6-362fcf9b8465 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.269487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.270328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.270816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.271372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.272281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.274460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.275737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.276631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.277960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.278353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.279105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.281315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.288441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 787/1588] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.289945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.299768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 802/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.300889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.302272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82411a59-9494-4d1e-94b7-d03df7a9953e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.307641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.308458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.308743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.309055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.309848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.312806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80018300-b121-43e4-96a4-f69f316ce45b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.313261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cb29a-4bd2-4765-b1c4-3b089356429a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.318990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.320126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.320562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.322288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.323648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.327363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.342203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 803/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.343580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.347110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 788/1591] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:13 2022] POST /identity/v3/users => generated 297 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:13.349107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.349483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.349836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.359676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab5910-13db-49bc-971e-c5a831234df7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.365678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.366699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.367140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.367808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.368645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.371242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.390590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] MFA Rules not processed for user `334506441b284e4c8472a391622da7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:13.392048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:13.408212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 804/1592] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:13 2022] POST /identity/v3/users => generated 301 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:13.410086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.433232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291043a-0929-4f67-b535-d51a502546f1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.440633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.441512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.441798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.442758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.443056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.444826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.461029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 805/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.463515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.476841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ab77c-e258-42dd-afc8-d35cd4002379 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.484022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.484985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.485430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.486113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.486980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.488804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.503858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 806/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.504908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.509104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 789/1595] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:13.510572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.516453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d5fef-56c0-4f1e-b542-5c3548a708b3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.524447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.525107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ca97c-306d-468c-aec8-ffb8d30d7ec5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.525677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.526174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.526645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.527475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.529344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.533605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.534262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.534552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.534826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.535474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.537154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.542900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 807/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.544052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.555669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b35383-94af-4da9-ba8f-b649f099d8ec admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.557845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 790/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.562875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.563909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.565340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.565924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.566358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] PATH_INFO: `/v3/projects/d16af15e86174ab1b9344d35ffe0844d/users/468522570ba7419c87da6d05fc0d1ff1/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.584306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe16040-faca-4ea2-ba4e-b2df8a2ec08b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.589329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16af15e86174ab1b9344d35ffe0844d, user_id=468522570ba7419c87da6d05fc0d1ff1, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.590736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:13.590736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:46:13.590736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:46:13.591046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.591745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.592442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.592853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.593575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.593729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.595843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.596745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:13.611125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 791/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:13.612019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.618217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 808/1599] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:13 2022] PUT /identity/v3/projects/d16af15e86174ab1b9344d35ffe0844d/users/468522570ba7419c87da6d05fc0d1ff1/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:13.620210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.631994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc32a2-941d-485c-bf52-61fb8b3b9654 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.719205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:13.742646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.743369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.743663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.744553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] PATH_INFO: `/v3/projects/1e3cef1ca6964f3dbae8e170ff530811/users/0fca332203b9473fa8b07286bc44300f/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.751060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4561c-7deb-4673-93da-94ce76d8058e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.758337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.759163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.759536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.759883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] PATH_INFO: `/v3/projects/3fd1bef8389445f0a8cfe492cbf22f8c/users/a0101b6f23d04016a89c986621ba5dde/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.769852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3cef1ca6964f3dbae8e170ff530811, user_id=0fca332203b9473fa8b07286bc44300f, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.772746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.786454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd1bef8389445f0a8cfe492cbf22f8c, user_id=a0101b6f23d04016a89c986621ba5dde, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.789478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.791589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 792/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:13 2022] PUT /identity/v3/projects/1e3cef1ca6964f3dbae8e170ff530811/users/0fca332203b9473fa8b07286bc44300f/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:13.795795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.808851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 809/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:13 2022] PUT /identity/v3/projects/3fd1bef8389445f0a8cfe492cbf22f8c/users/a0101b6f23d04016a89c986621ba5dde/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:13.809530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.900928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d7213-62d9-4197-9145-0fc3e0cdf76a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.906315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.906958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.907217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.907511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.908139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.909314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabd7ed-e04f-4ddf-99f3-1fdc3deaed62 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:13.910130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.914032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:13.914690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.914968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.915297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] PATH_INFO: `/v3/projects/862dd2125cb94b379981a70b191449dd/users/f8abe86bffe649c1a96d76ed6d40b430/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.931883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862dd2125cb94b379981a70b191449dd, user_id=f8abe86bffe649c1a96d76ed6d40b430, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:13.934604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:13.940429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 793/1602] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:13 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:13.943871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.956688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 810/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:13 2022] PUT /identity/v3/projects/862dd2125cb94b379981a70b191449dd/users/f8abe86bffe649c1a96d76ed6d40b430/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:13.957780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.022833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c909b-5d30-4265-b31f-594795fedd13 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.027822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.028454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.028743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.028999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609fb1-c37b-40fd-bea2-de452f7783a7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.029120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.029689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.031305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08c6e0-d87a-4588-8ddf-fcbc0885b311 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.035067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.035910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.036339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.036873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.037791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.039568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70387e31-f97e-4dd9-bd3d-7773682bbde1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.041700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:14.043889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 794/1604] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:14.045297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6b7b-1490-4587-9994-ed8fe4fdb4a1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.045592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6b7b-1490-4587-9994-ed8fe4fdb4a1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.045872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6b7b-1490-4587-9994-ed8fe4fdb4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.058222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 811/1605] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:14.059975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.060413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.060852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.067381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a1fe6b7b-1490-4587-9994-ed8fe4fdb4a1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.068612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a1fe6b7b-1490-4587-9994-ed8fe4fdb4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.088447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.089737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.092472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 795/1606] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.098112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea821b2-fb62-4f24-8b24-fb3a1ef2326e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.098112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea821b2-fb62-4f24-8b24-fb3a1ef2326e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.098112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea821b2-fb62-4f24-8b24-fb3a1ef2326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.104870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 812/1607] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.105798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.116603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9697-7f2b-4298-858b-2680b8349df4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.121230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.121840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.122122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.122393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.123021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.124972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.132525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-bea821b2-fb62-4f24-8b24-fb3a1ef2326e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.134455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bea821b2-fb62-4f24-8b24-fb3a1ef2326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.144123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 796/1608] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.145536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.145824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.146104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.152777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 813/1609] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:14.153976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.166414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9315-ab91-4d05-8c75-b275801cd357 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.171753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.172426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.172709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.173125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] PATH_INFO: `/v3/projects/089a845bce2542969b7e6f013b28d928/users/79acdbc2c11c4459928eab5b83a6755d/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.173253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.175068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.183441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 797/1610] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.184128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.193312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=089a845bce2542969b7e6f013b28d928, user_id=79acdbc2c11c4459928eab5b83a6755d, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.196275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.200440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71853d7-a6ee-4a34-9b91-97ab5f83c7ce None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.205763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.206437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.206717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.206993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] PATH_INFO: `/v3/projects/38a1a852435245968f57478a4d1a4757/users/336bfea93e734360847f8ed3b9b0af10/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.210375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:14.218204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 814/1611] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:14 2022] PUT /identity/v3/projects/089a845bce2542969b7e6f013b28d928/users/79acdbc2c11c4459928eab5b83a6755d/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:14.218204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.224150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a1a852435245968f57478a4d1a4757, user_id=336bfea93e734360847f8ed3b9b0af10, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.227355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.244098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 798/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:14 2022] PUT /identity/v3/projects/38a1a852435245968f57478a4d1a4757/users/336bfea93e734360847f8ed3b9b0af10/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:14.245192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.294575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3284e39-f30c-462e-9252-4d5bfa4d8c5c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.315333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9719d-de17-4289-bd20-e602b50b4a83 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.319789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.321272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.321555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.321831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.356893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.357572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.358032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.358269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.415198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.416463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.435268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 799/1613] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:14.437461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.449709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d883d0-73ea-4ec5-a713-de3186082658 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.453844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.454546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.454901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.455047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.455447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.456776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.471713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 815/1614] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:14.472998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.474226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.475909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.485835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050bad9-76db-443d-9f95-16eebfd692e9 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.492371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.493295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.493688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.494061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.494179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 800/1615] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:14.494900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.496482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.496788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fc8f8-0b03-4ad8-86fb-29e0ff26a5da admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.513687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64447f4-4522-4978-ba99-dcfcf048120c service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.517181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 816/1616] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:14.518320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.518668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.519065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.519193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.519348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.519605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.519736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.530955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:14.543574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.545067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180cb90-c9b1-42a5-b39c-4c8f2d38bda4 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.556010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.558548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 801/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:14.562485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.562737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.562737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.562737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.583218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 817/1618] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.583218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.604246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d06eb-f665-4a0e-843a-1dad4a847ed5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.616002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.617844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.618132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.618570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] PATH_INFO: `/v3/projects/c17410fa824a4e80be7ea19c30af99f8/users/a33234164a924cb7b125597a324e395b/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.630256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] MFA Rules not processed for user `336bfea93e734360847f8ed3b9b0af10`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:14.631513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.652564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17410fa824a4e80be7ea19c30af99f8, user_id=a33234164a924cb7b125597a324e395b, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.660493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.688842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 818/1619] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:14 2022] PUT /identity/v3/projects/c17410fa824a4e80be7ea19c30af99f8/users/a33234164a924cb7b125597a324e395b/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:14.688842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.767948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bb0c7-8799-40ae-a5fa-fae02c378c63 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.773052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.773988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.774659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.775187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.791008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:14.812080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 802/1620] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:14.815687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.846676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80669d8-1f80-40aa-bdb1-c9584c595cd7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.852927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.855032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.859949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.860602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.860891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.861219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.863259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.867625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.870209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 819/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:14.871350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.885095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb9e53-c192-40da-b99f-491b0a065cca service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.894864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.895645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.895994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.896409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.897198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.899404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.912585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 803/1622] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:14.913597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.934722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a978afb-6eb2-44dc-8b00-a39a588b01e2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.940969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.941752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.942033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.942313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] PATH_INFO: `/v3/projects/c17410fa824a4e80be7ea19c30af99f8/users/a33234164a924cb7b125597a324e395b/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.945603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17410fa824a4e80be7ea19c30af99f8, user_id=a33234164a924cb7b125597a324e395b, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.950095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 820/1623] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/projects => generated 397 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:14.951745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337805e6-e32c-4534-a675-0446ee4c5c99 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.954515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:14.983296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 804/1624] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:14 2022] PUT /identity/v3/projects/c17410fa824a4e80be7ea19c30af99f8/users/a33234164a924cb7b125597a324e395b/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:14.997679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.999093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.999385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.070235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] MFA Rules not processed for user `a33234164a924cb7b125597a324e395b`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:15.072292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:15.086424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45460f43-0b6f-4c1e-8a64-101726bce000 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.091378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:15.187813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.188700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.190876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.191359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.192934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.199308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.245744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 805/1625] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:15.258697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 821/1626] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:14 2022] POST /identity/v3/users => generated 305 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:15.262029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.263292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.285183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066555-94f5-4a7b-8119-ae318831b97f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.302109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.302932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.303312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.303684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.304899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.308141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.331519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 822/1627] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:15.337485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.364915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce17d07-f8ac-4d3a-a269-b3cfeca2275f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.377900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.378874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.379231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.379578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] PATH_INFO: `/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b/users/014d4b0cbd5a418f8729026451a56f6d/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.428038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdacfa88eae3489d97f3f9a100bb8a3b, user_id=014d4b0cbd5a418f8729026451a56f6d, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.431531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701fa6-8a8b-4e49-875d-fefe7e45487f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.433841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.449723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.450384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.450668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.450947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.473476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 823/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:15 2022] PUT /identity/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b/users/014d4b0cbd5a418f8729026451a56f6d/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:15.477523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.480891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.482371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.504461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 806/1629] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:46:15.615184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3a563-c2d8-416e-88b9-3a5c40f3a69e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.632235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.633632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.633933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.634583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.636211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.641455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.672202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.674701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 824/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:15.678502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.684331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c754d35-788c-4159-91e5-18e9f91e7ddb service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.701740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c5-4054-4f8e-be60-f59ba76cdbf1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:15.707315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.708335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.708617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.709028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] PATH_INFO: `/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b/users/014d4b0cbd5a418f8729026451a56f6d/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.711710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdacfa88eae3489d97f3f9a100bb8a3b, user_id=014d4b0cbd5a418f8729026451a56f6d, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:15.715907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78621c-c63c-43c5-8ba0-d74c2c0fd6b2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:15.733319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 825/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:15 2022] PUT /identity/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b/users/014d4b0cbd5a418f8729026451a56f6d/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:15.743776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.744070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.744350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.779540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] MFA Rules not processed for user `014d4b0cbd5a418f8729026451a56f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:15.780542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:15.835684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:15.875808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:15.882513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:15.884996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:15.885462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:15.912608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 826/1632] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:15.930433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:15.947303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:16.071365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2b5b-d2f3-41bb-8f85-f80562985c3a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:16.081518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:16.084559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:16.085953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:16.086667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:16.105267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:16.112995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:16.126760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:16.130594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef987b-1cf4-4618-9db7-d3acf0013ce2 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:16.153897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 807/1633] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:16.168436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 827/1634] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:15 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:16.336912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:16.350764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34304155-374a-48f6-95ce-fbc8b9495c84 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:16.356004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:16.356920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:16.357378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:16.357952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:16.463050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:16.465089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:16.496308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 808/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:16 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:46:17.985892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:17.988004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:17.988485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.068586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] MFA Rules not processed for user `f8abe86bffe649c1a96d76ed6d40b430`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:18.077817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:18.299679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 828/1636] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:18.304380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:18.462522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d20f-1ac3-43a0-9fee-88060e1e45db service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:18.470075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:18.470918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.471303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.471658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.472513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:18.474739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:18.530210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 809/1637] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:18 2022] POST /identity/v3/projects => generated 405 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:18.534149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:18.549582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e53a9c-f6f7-40a7-8cd1-591d5a028cd8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:18.672882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:18.675093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.675802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.677370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.678363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:18.684989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:18.728507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:18.781676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 829/1638] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:18 2022] POST /identity/v3/users => generated 309 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:18.786393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:18.805433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2302f-4e8d-4bfa-8be8-373419db6b4c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:18.821762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:18.822462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.822753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.825926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.827952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:18.831894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:18.890105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 810/1639] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:18.895324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:18.909927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa059-8f7a-4a21-b50c-ea90891c36f0 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:18.924663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:18.925874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:18.926162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:18.926438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] PATH_INFO: `/v3/projects/4aa60b53ed014cf68a10001da5bf10cf/users/08df4a5eed0346d78a1ac63b0b0ef762/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:18.964521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa60b53ed014cf68a10001da5bf10cf, user_id=08df4a5eed0346d78a1ac63b0b0ef762, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:18.969733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:19.007650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 830/1640] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:18 2022] PUT /identity/v3/projects/4aa60b53ed014cf68a10001da5bf10cf/users/08df4a5eed0346d78a1ac63b0b0ef762/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:19.011545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:19.028907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904ae74-6aa3-4ac2-8fd0-c8e8c53a360f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:19.128083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:19.271207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:19.272267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:19.272764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:19.273477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:19.291819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:19.295964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bdd91-0fe3-4ce9-9e38-6aed59f54078 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:19.323885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 811/1641] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:19.328441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:19.371507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d54bb-9ca9-40bf-ab1f-d0ce161094f6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:19.383083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:19.384469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:19.385270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:19.386062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] PATH_INFO: `/v3/projects/4aa60b53ed014cf68a10001da5bf10cf/users/08df4a5eed0346d78a1ac63b0b0ef762/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:19.411449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa60b53ed014cf68a10001da5bf10cf, user_id=08df4a5eed0346d78a1ac63b0b0ef762, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:19.425305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53858617-2108-4fb2-9227-f0b200ecec89 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:19.469696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 831/1642] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:19 2022] PUT /identity/v3/projects/4aa60b53ed014cf68a10001da5bf10cf/users/08df4a5eed0346d78a1ac63b0b0ef762/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:20.185058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b58ad-d324-41ff-ad12-e1f09ee1f18e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:20.186200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b58ad-d324-41ff-ad12-e1f09ee1f18e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:20.186687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b58ad-d324-41ff-ad12-e1f09ee1f18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:20.228608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a30b58ad-d324-41ff-ad12-e1f09ee1f18e None None] MFA Rules not processed for user `5b9f29fab9e24e98a632416f2160182c`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:20.230309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a30b58ad-d324-41ff-ad12-e1f09ee1f18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:20.347750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 812/1643] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:20.349642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f935e-547e-425b-a14f-f1bff652b19b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:20.350013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f935e-547e-425b-a14f-f1bff652b19b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:20.350192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f935e-547e-425b-a14f-f1bff652b19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:20.370715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-040f935e-547e-425b-a14f-f1bff652b19b None None] MFA Rules not processed for user `5b9f29fab9e24e98a632416f2160182c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:20.371952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-040f935e-547e-425b-a14f-f1bff652b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:20.384839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 832/1644] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:20.409612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112462af-71c8-483a-aca0-a53d364515ac None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:20.409913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112462af-71c8-483a-aca0-a53d364515ac None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:20.410199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112462af-71c8-483a-aca0-a53d364515ac None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:20.411710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 813/1645] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 31 09:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:20.432559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5593d34-6f4e-4cce-8cbb-af1adc8e50ee None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:20.432848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5593d34-6f4e-4cce-8cbb-af1adc8e50ee None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:20.433156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5593d34-6f4e-4cce-8cbb-af1adc8e50ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:20.515635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d5593d34-6f4e-4cce-8cbb-af1adc8e50ee None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:20.517224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d5593d34-6f4e-4cce-8cbb-af1adc8e50ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:20.625630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 833/1646] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 31 09:46:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:20.634575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112462af-71c8-483a-aca0-a53d364515ac None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:20.650431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112462af-71c8-483a-aca0-a53d364515ac None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:20.657854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:20.658084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:20.658402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:20.658682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:20.669866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:20.672905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd819c-9a1b-4498-a018-df139d881fe8 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:20.692334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 814/1647] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:21.076541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:21.076847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:21.077135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:21.078600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 834/1648] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 31 09:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:21.087251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c209e4-d68b-4b95-b9b0-4387d59775e0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:21.087251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c209e4-d68b-4b95-b9b0-4387d59775e0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:21.087251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c209e4-d68b-4b95-b9b0-4387d59775e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:21.121363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f5c209e4-d68b-4b95-b9b0-4387d59775e0 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:21.122618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f5c209e4-d68b-4b95-b9b0-4387d59775e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:21.134663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 815/1649] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:46:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:21.160530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:21.162516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:21.374497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be0365-03f3-4165-b75a-9af4a6481038 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:21.382593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:21.383476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:21.383839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:21.384189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:21.398422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:21.400178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f358e-d920-4786-896d-f13baebd58c7 service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:21.439926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 835/1650] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:21.937605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09257-a158-4226-b05f-3727a11999f6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:21.938037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09257-a158-4226-b05f-3727a11999f6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:21.938253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09257-a158-4226-b05f-3727a11999f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:21.983799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e4e09257-a158-4226-b05f-3727a11999f6 None None] MFA Rules not processed for user `08df4a5eed0346d78a1ac63b0b0ef762`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:21.985922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e4e09257-a158-4226-b05f-3727a11999f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:22.140836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 816/1651] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:22.576886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3c044-912a-4fa3-9bc9-8e561ee1f580 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:22.577379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3c044-912a-4fa3-9bc9-8e561ee1f580 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:22.577458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3c044-912a-4fa3-9bc9-8e561ee1f580 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:22.579074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 836/1652] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 31 09:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:22.587315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:22.588126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:22.588691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:22.692553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:22.735432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:22.743723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:22.885904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 817/1653] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:46:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:22.958847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3c044-912a-4fa3-9bc9-8e561ee1f580 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:22.975295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3c044-912a-4fa3-9bc9-8e561ee1f580 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:22.984060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:22.986277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:22.986764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:22.987242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:22.995184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:23.001483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:23.004821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd782a-3201-4513-ad79-0dcfecfdd940 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:23.024514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 837/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:24.621542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:24.859506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33493078-1fd5-409f-b89c-7e5bd9b9621d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:24.894867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:24.895376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:24.895656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:24.895913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:24.903962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:24.945501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:24.947216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722d030-edda-4f71-b253-3fef2e4927fd service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:24.982026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 818/1655] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:25.261535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da481d-a520-401e-b76f-9acb76094412 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:25.262123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da481d-a520-401e-b76f-9acb76094412 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:25.263124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da481d-a520-401e-b76f-9acb76094412 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:25.265873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 838/1656] 10.222.0.25 () {54 vars in 962 bytes} [Wed Aug 31 09:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:25.283803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd3a6-d516-4680-b9c0-4ad60330544e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:25.284474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd3a6-d516-4680-b9c0-4ad60330544e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:25.285043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cd3a6-d516-4680-b9c0-4ad60330544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:25.335143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ef1cd3a6-d516-4680-b9c0-4ad60330544e None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:25.340398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ef1cd3a6-d516-4680-b9c0-4ad60330544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:25.367213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 819/1657] 10.222.0.25 () {58 vars in 1059 bytes} [Wed Aug 31 09:46:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:25.372498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b00d-1f59-43fb-aece-ba9af80ea72b None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:25.372938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b00d-1f59-43fb-aece-ba9af80ea72b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:25.373371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b00d-1f59-43fb-aece-ba9af80ea72b None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:25.375245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 839/1658] 10.222.0.25 () {54 vars in 912 bytes} [Wed Aug 31 09:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:25.382199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403f38-2796-4fc9-852c-25661cd9463c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:25.382741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403f38-2796-4fc9-852c-25661cd9463c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:25.383234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403f38-2796-4fc9-852c-25661cd9463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:25.433754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-88403f38-2796-4fc9-852c-25661cd9463c None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:25.443807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-88403f38-2796-4fc9-852c-25661cd9463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:25.470382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 820/1659] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:46:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:25.614630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b00d-1f59-43fb-aece-ba9af80ea72b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:25.629976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b00d-1f59-43fb-aece-ba9af80ea72b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:25.644086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:25.644885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:25.645328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:25.645705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:25.664449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:25.666176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2919c8-6026-4714-935d-639458d7bc24 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:25.701404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 840/1660] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:26.270176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d339f2-6353-47f3-b82f-023f4e3b3094 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:26.270645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d339f2-6353-47f3-b82f-023f4e3b3094 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:26.270645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d339f2-6353-47f3-b82f-023f4e3b3094 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:26.272584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 821/1661] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:26.309749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe739-77a8-4759-b0fb-37842feec467 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:26.310244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe739-77a8-4759-b0fb-37842feec467 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:26.310360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe739-77a8-4759-b0fb-37842feec467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:26.351178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-72cbe739-77a8-4759-b0fb-37842feec467 None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:26.352422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-72cbe739-77a8-4759-b0fb-37842feec467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:26.377454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 841/1662] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:46:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:26.411597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb49ea-6dae-4641-b0df-c111f14a0d60 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:26.411880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb49ea-6dae-4641-b0df-c111f14a0d60 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:26.412153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb49ea-6dae-4641-b0df-c111f14a0d60 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:26.413880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 822/1663] 10.222.0.25 () {54 vars in 961 bytes} [Wed Aug 31 09:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:26.419963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:26.420247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:26.427585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:26.469654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:26.470932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:26.491948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 842/1664] 10.222.0.25 () {58 vars in 1058 bytes} [Wed Aug 31 09:46:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:26.503633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb49ea-6dae-4641-b0df-c111f14a0d60 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:26.532794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb49ea-6dae-4641-b0df-c111f14a0d60 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:26.548362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:26.549693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:26.550080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:26.550464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:26.572781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:26.574605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a41df-b44c-4d69-9563-621252f3d790 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:26.615631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 823/1665] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:28.514745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:28.757560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438fb9d-c913-47c9-9ddc-0b36f7caa074 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:28.788186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:28.790259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:28.790563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:28.790986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] PATH_INFO: `/v3/users/336bfea93e734360847f8ed3b9b0af10` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:28.797132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336bfea93e734360847f8ed3b9b0af10)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:28.798640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: WARNING py.warnings [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:28.798640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: warnings.warn(msg) Aug 31 09:46:28.798640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: Aug 31 09:46:28.801269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:29.058735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336bfea93e734360847f8ed3b9b0af10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:29.061493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336bfea93e734360847f8ed3b9b0af10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:29.115602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336bfea93e734360847f8ed3b9b0af10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:29.164573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336bfea93e734360847f8ed3b9b0af10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:29.187013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae3e6588-7e31-4197-a17d-2d88ea6569a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336bfea93e734360847f8ed3b9b0af10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:29.197117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 843/1666] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:28 2022] DELETE /identity/v3/users/336bfea93e734360847f8ed3b9b0af10 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:29.202819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.203355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.203725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.247846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:29.253703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:29.400319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8af11-719e-4368-8b54-5cf2095b4f87 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.400319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8af11-719e-4368-8b54-5cf2095b4f87 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.400805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8af11-719e-4368-8b54-5cf2095b4f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.418831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 824/1667] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:29.606598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-25c8af11-719e-4368-8b54-5cf2095b4f87 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:29.608135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-25c8af11-719e-4368-8b54-5cf2095b4f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:29.634281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:29.637341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:29.696872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:29.719962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:29.820564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c562f-1028-48e1-9ad7-fe6c0309993b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:29.838369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:29.838998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.839279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.839550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.859475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 844/1668] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:29.862447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:29.862903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:29.863320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:29.868053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:29.872157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:29.911799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 825/1669] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:46:29.953847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] MFA Rules not processed for user `468522570ba7419c87da6d05fc0d1ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:29.960189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:30.214048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 845/1670] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:30.235001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:30.238097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:30.238259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:30.239977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:30.420285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2bb72-8d04-46a0-91fb-f9e3867015b6 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:30.432892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:30.433555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.433833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.434108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.454086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:30.456287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:30.481959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 846/1671] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:30.584733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e52ad1-4b77-4b93-9484-b4ba449a6097 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:30.597329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:30.598080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.598611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.599000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.620214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:30.622043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:30.673740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 826/1672] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:46:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:46:30.759529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:30.763277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:30.832587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f7b85-94b4-401a-98d5-ea7e06259ae1 service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:30.865912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:30.866911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:30.875992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:30.876489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] PATH_INFO: `/v3/users/f8abe86bffe649c1a96d76ed6d40b430` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:30.905383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8abe86bffe649c1a96d76ed6d40b430)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:30.937547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:31.117395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:31.167675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8abe86bffe649c1a96d76ed6d40b430'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:31.169547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8abe86bffe649c1a96d76ed6d40b430'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:31.229207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8abe86bffe649c1a96d76ed6d40b430'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:31.256772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8abe86bffe649c1a96d76ed6d40b430'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:31.269087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea7d743b-85b4-4030-8a0d-be6ad91214f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8abe86bffe649c1a96d76ed6d40b430'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:31.294688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 847/1673] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:30 2022] DELETE /identity/v3/users/f8abe86bffe649c1a96d76ed6d40b430 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:31.786129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:31.788982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:31.838406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f391-c567-4853-bea1-e7c7f786dbfd service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:32.126019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:32.126953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:32.127390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:32.127836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] PATH_INFO: `/v3/users/334506441b284e4c8472a391622da7a6` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:32.133186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=334506441b284e4c8472a391622da7a6)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:32.153057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:32.265517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:32.295624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334506441b284e4c8472a391622da7a6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:32.327682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334506441b284e4c8472a391622da7a6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:32.378813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334506441b284e4c8472a391622da7a6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:32.413670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334506441b284e4c8472a391622da7a6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:32.416035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334506441b284e4c8472a391622da7a6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:32.438068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 827/1674] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:31 2022] DELETE /identity/v3/users/334506441b284e4c8472a391622da7a6 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:32.444088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:32.444975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:32.445269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:32.496719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:32.502318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:32.783757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 848/1675] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:32.817503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:32.820533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:32.852409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2ded2-b435-4b37-bb7a-20506347deeb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:32.866203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:32.867819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:32.868286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:32.868776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] PATH_INFO: `/v3/projects/862dd2125cb94b379981a70b191449dd` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:32.870515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] RBAC: Authorizing `identity:delete_project(project_id=862dd2125cb94b379981a70b191449dd)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:32.872022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: WARNING py.warnings [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:46:32.872022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: warnings.warn(msg) Aug 31 09:46:32.872022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: Aug 31 09:46:32.874074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:32.920021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:32.923991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:33.117712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862dd2125cb94b379981a70b191449dd', 'request_id': 'req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.119642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ea92-f70a-4d7a-9334-c6e934890d4b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:33.149413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862dd2125cb94b379981a70b191449dd', 'request_id': 'req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.194526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862dd2125cb94b379981a70b191449dd', 'request_id': 'req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.269608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] The token cache is being invalidate because project 862dd2125cb94b379981a70b191449dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:33.269892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.274000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.277410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 828/1676] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:32 2022] DELETE /identity/v3/projects/862dd2125cb94b379981a70b191449dd => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:33.281716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:33.289191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:33.289493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:33.290764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:33.291315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:33.291808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:33.294613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:33.348695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:33.528097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:33.529831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:33.544224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18cdc7-6932-4e81-bfee-dc30d1f6effa admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:33.565707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 849/1677] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:33.566865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:33.570428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:33.572351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:33.573110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:33.573530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:33.573885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] PATH_INFO: `/v3/users/08df4a5eed0346d78a1ac63b0b0ef762` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:33.581170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08df4a5eed0346d78a1ac63b0b0ef762)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:33.581972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:33.606420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed901161-30cd-4020-bcc2-b891114f92f5 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:33.622003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:33.623264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:33.623691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:33.624078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] PATH_INFO: `/v3/projects/38a1a852435245968f57478a4d1a4757` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:33.626460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a1a852435245968f57478a4d1a4757)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:33.634400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:33.697700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:33.707571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a1a852435245968f57478a4d1a4757', 'request_id': 'req-41b770e6-a349-4e6f-a674-4c6975f90ec9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.714994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a1a852435245968f57478a4d1a4757', 'request_id': 'req-41b770e6-a349-4e6f-a674-4c6975f90ec9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.724794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a1a852435245968f57478a4d1a4757', 'request_id': 'req-41b770e6-a349-4e6f-a674-4c6975f90ec9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.730528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08df4a5eed0346d78a1ac63b0b0ef762'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.747008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] The token cache is being invalidate because project 38a1a852435245968f57478a4d1a4757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:33.747967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.749073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.751603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 850/1678] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:33 2022] DELETE /identity/v3/projects/38a1a852435245968f57478a4d1a4757 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:33.755532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:33.758374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:33.761823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08df4a5eed0346d78a1ac63b0b0ef762'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.786721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08df4a5eed0346d78a1ac63b0b0ef762'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.801296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08df4a5eed0346d78a1ac63b0b0ef762'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.802371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d77fdb0c-6824-47e8-a3a1-2c0043dbbd71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08df4a5eed0346d78a1ac63b0b0ef762'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:33.816344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 829/1679] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:33 2022] DELETE /identity/v3/users/08df4a5eed0346d78a1ac63b0b0ef762 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:33.866737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b770e6-a349-4e6f-a674-4c6975f90ec9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:33.974144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:34.012893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:34.013590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:34.013864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:34.014134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] PATH_INFO: `/v3/users/014d4b0cbd5a418f8729026451a56f6d` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:34.015646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=014d4b0cbd5a418f8729026451a56f6d)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:34.018204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:34.282106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014d4b0cbd5a418f8729026451a56f6d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:34.285874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014d4b0cbd5a418f8729026451a56f6d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:34.384767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014d4b0cbd5a418f8729026451a56f6d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:34.404730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014d4b0cbd5a418f8729026451a56f6d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:34.436429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014d4b0cbd5a418f8729026451a56f6d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:34.459315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 851/1680] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:33 2022] DELETE /identity/v3/users/014d4b0cbd5a418f8729026451a56f6d => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:35.994827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900332e-b510-4c89-88d8-c5842f1b7c5b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:35.996192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900332e-b510-4c89-88d8-c5842f1b7c5b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:35.997082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900332e-b510-4c89-88d8-c5842f1b7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:36.045338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d900332e-b510-4c89-88d8-c5842f1b7c5b None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:36.046862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d900332e-b510-4c89-88d8-c5842f1b7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:36.173679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 830/1681] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Aug 31 09:46:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:36.258915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:36.262043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:36.299078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514c27-5f21-47d5-93e3-e191135a84bf admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:36.309997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:36.310906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:36.311370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:36.314542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:36.338521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:36.340364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:36.354083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 852/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:36.784974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e3fe-6da6-4047-a4c5-72f0f20fe7d4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:36.785394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e3fe-6da6-4047-a4c5-72f0f20fe7d4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:36.785667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e3fe-6da6-4047-a4c5-72f0f20fe7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:36.858446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b9d6e3fe-6da6-4047-a4c5-72f0f20fe7d4 None None] MFA Rules not processed for user `79acdbc2c11c4459928eab5b83a6755d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:36.862598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b9d6e3fe-6da6-4047-a4c5-72f0f20fe7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:36.913327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:36.914995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.110134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 831/1683] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:37.120065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.120569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.121035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.178860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] MFA Rules not processed for user `79acdbc2c11c4459928eab5b83a6755d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:37.181287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.248874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 832/1684] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:37.260034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:37.261204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bafdc-d2e5-4d9f-a53c-a66be871701e service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:37.263463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.270351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:37.272225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.272705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.273317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] PATH_INFO: `/v3/projects/4aa60b53ed014cf68a10001da5bf10cf` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.274886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa60b53ed014cf68a10001da5bf10cf)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:37.277788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:37.435858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa60b53ed014cf68a10001da5bf10cf', 'request_id': 'req-11e0ed2e-1449-4848-a2d4-fb30b06214d8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:37.442899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:37.455720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa60b53ed014cf68a10001da5bf10cf', 'request_id': 'req-11e0ed2e-1449-4848-a2d4-fb30b06214d8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:37.472300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa60b53ed014cf68a10001da5bf10cf', 'request_id': 'req-11e0ed2e-1449-4848-a2d4-fb30b06214d8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:37.500726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] The token cache is being invalidate because project 4aa60b53ed014cf68a10001da5bf10cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:37.501072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:37.505487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:37.507107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:37.512877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 853/1685] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:36 2022] DELETE /identity/v3/projects/4aa60b53ed014cf68a10001da5bf10cf => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:37.512877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:37.512877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.596396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4633-bbf4-4764-852c-b03ed380a21c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:37.730332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:37.731136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.731846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.732133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.735328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:37.749602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ed2e-1449-4848-a2d4-fb30b06214d8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:37.781628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:37.782368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:37.783855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:37.786776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] PATH_INFO: `/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:37.790123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdacfa88eae3489d97f3f9a100bb8a3b)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:37.792103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:37.976912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:37.983107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30ff2f-f701-4fd4-89b6-9506ee289aae service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:38.008821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdacfa88eae3489d97f3f9a100bb8a3b', 'request_id': 'req-be4c0caf-f545-44af-9ed7-37b2b5097598'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.029679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdacfa88eae3489d97f3f9a100bb8a3b', 'request_id': 'req-be4c0caf-f545-44af-9ed7-37b2b5097598'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.045476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 833/1686] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:38.048042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82131-9b81-41aa-acfe-737e60462b20 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:38.055705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82131-9b81-41aa-acfe-737e60462b20 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:38.060917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82131-9b81-41aa-acfe-737e60462b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:38.090936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdacfa88eae3489d97f3f9a100bb8a3b', 'request_id': 'req-be4c0caf-f545-44af-9ed7-37b2b5097598'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.145272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] The token cache is being invalidate because project fdacfa88eae3489d97f3f9a100bb8a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:38.145577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.154402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.158604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 854/1687] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:37 2022] DELETE /identity/v3/projects/fdacfa88eae3489d97f3f9a100bb8a3b => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:38.159215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:38.166143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ccf82131-9b81-41aa-acfe-737e60462b20 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:38.167507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ccf82131-9b81-41aa-acfe-737e60462b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:38.171239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:38.479764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 834/1688] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:38.487835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecaddf-5861-4b82-9814-531e8b8cb29f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:38.488152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecaddf-5861-4b82-9814-531e8b8cb29f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:38.488377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecaddf-5861-4b82-9814-531e8b8cb29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:38.499860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c0caf-f545-44af-9ed7-37b2b5097598 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:38.549750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:38.554626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:38.554902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:38.555139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ccecaddf-5861-4b82-9814-531e8b8cb29f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:38.555251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] PATH_INFO: `/v3/projects/0db155f32ed94921a444706298b1f2d9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:38.556385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ccecaddf-5861-4b82-9814-531e8b8cb29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:38.557723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db155f32ed94921a444706298b1f2d9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:38.563894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:38.590474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 835/1689] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:38.591755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bf1ec-1738-44f2-8521-b4e3f128ac80 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:38.592087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bf1ec-1738-44f2-8521-b4e3f128ac80 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:38.592307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bf1ec-1738-44f2-8521-b4e3f128ac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:38.614658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:38.641548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6e4bf1ec-1738-44f2-8521-b4e3f128ac80 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:38.643304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6e4bf1ec-1738-44f2-8521-b4e3f128ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:38.687523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 836/1690] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:38.689606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:38.689951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:38.690188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:38.729660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db155f32ed94921a444706298b1f2d9', 'request_id': 'req-d67dceae-de11-4fb8-acee-77f54168c410'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.749914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db155f32ed94921a444706298b1f2d9', 'request_id': 'req-d67dceae-de11-4fb8-acee-77f54168c410'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.758332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:38.759592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:38.770595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db155f32ed94921a444706298b1f2d9', 'request_id': 'req-d67dceae-de11-4fb8-acee-77f54168c410'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.783485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:38.825178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] The token cache is being invalidate because project 0db155f32ed94921a444706298b1f2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:38.825875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.827448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:38.832848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 855/1691] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:38 2022] DELETE /identity/v3/projects/0db155f32ed94921a444706298b1f2d9 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:38.835534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:38.839750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:39.006421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 837/1692] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:39.007394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.008733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:39.035797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f881f8e-597b-48b5-8828-55ad2f567989 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.046957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.048410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.048860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.049302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] PATH_INFO: `/v3/users/a33234164a924cb7b125597a324e395b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.053046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33234164a924cb7b125597a324e395b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.053831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dceae-de11-4fb8-acee-77f54168c410 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.057421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:39.060595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.061618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.062344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.062344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.063144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:39.063616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.067144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afaff91-7ae9-4a08-8910-fecadbfe827f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:39.100538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 856/1693] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:39.100538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.100538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.100538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.189579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] MFA Rules not processed for user `a0101b6f23d04016a89c986621ba5dde`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:39.205312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33234164a924cb7b125597a324e395b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:39.211786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:39.223051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33234164a924cb7b125597a324e395b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:39.253493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33234164a924cb7b125597a324e395b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:39.272766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33234164a924cb7b125597a324e395b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:39.273758 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33234164a924cb7b125597a324e395b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:39.295496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 838/1694] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:39 2022] DELETE /identity/v3/users/a33234164a924cb7b125597a324e395b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:39.296422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.298413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:39.483343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cfcd9-27b2-4eae-b0db-249fe0dbd8b1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.499916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 857/1695] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:39.502398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.508276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:39.518505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.519485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.519774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.520048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.520718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:39.521068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.522703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:39.552185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da91e3-d1f8-4547-8959-b39416f11707 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.567300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.568339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.568881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.568881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.569414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.572542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:39.589943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 839/1696] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:39.590934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.611922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5245fd-4b19-4d22-867b-33d868713e0d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.637182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.637716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.638000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.638406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.639141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.646367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:39.682662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 858/1697] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:39 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:39.686833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.700158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:39.703206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:39.766435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 840/1698] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:39 2022] POST /identity/v3/projects => generated 389 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:39.770887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:39.881899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1418-f158-47bb-9a66-fbc5fc9150a0 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.949582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dadde0-1d8e-449e-bf4f-bb1a2a2afee0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:39.974468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:39.975167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:39.975464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:39.976839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:39.977594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:39.986100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7b741-5ac0-4fdf-bc87-c51b6f1d3196 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.040495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.041055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.041340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.041613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.042548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.044501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.045721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 841/1699] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:39 2022] POST /identity/v3/users => generated 295 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:46:40.047387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.047698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.048064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.100727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 859/1700] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:39 2022] POST /identity/v3/users => generated 289 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:40.100727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.128997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65491c1-cb78-4455-a281-0b39d55ca3b3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.136151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.136856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.137196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.137475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.137800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] MFA Rules not processed for user `0fca332203b9473fa8b07286bc44300f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:40.138253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.140525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.142531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:40.214496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 860/1701] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:40.218831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.248982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047e207-75f4-4961-8c96-f01786d6d359 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.272747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.275319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.275619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.275891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.276558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.281410 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.317136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 861/1702] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:40.318359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.344560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bdda9-9336-4090-bb47-7d51ac51c772 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.368664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.368664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.368664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.368664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.369260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.369260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.388787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 862/1703] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:40.388787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.390550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:40.441359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 842/1704] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:40.441359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.482924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502a590-2fc2-43e7-a0ec-db5648c37f45 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.498721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.499664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.500069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.500395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.501803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.507856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.542415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 843/1705] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:40.550864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.573244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:40.586174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bc41c-f790-4af1-a19f-1e01b3681d3b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.597413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e7a0-705b-46d4-8d6a-429335e3babe admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:40.607844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.608660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.609063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.609428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] PATH_INFO: `/v3/projects/83f5ad9dfb304c5c903d8c1e9e81d63f/users/0b262e34779546fdb0c773cf81a81f1b/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.610954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:40.611636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.612110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.612791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.670150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f5ad9dfb304c5c903d8c1e9e81d63f, user_id=0b262e34779546fdb0c773cf81a81f1b, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.679947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7bafc-6741-45a2-a1d6-5f35bf815c4f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.755635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:40.759612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac4035-1a8a-48ac-90fc-d2f8c163c340 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:40.761189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 844/1706] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:40 2022] PUT /identity/v3/projects/83f5ad9dfb304c5c903d8c1e9e81d63f/users/0b262e34779546fdb0c773cf81a81f1b/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:40.770610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c1f63-f7bf-475f-90c0-ec4273b975cc None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.770610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c1f63-f7bf-475f-90c0-ec4273b975cc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.770610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c1f63-f7bf-475f-90c0-ec4273b975cc None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.773561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 863/1707] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:40.775042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a1b2-52e7-4063-add7-89f58ebc2999 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.775420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a1b2-52e7-4063-add7-89f58ebc2999 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.775706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a1b2-52e7-4063-add7-89f58ebc2999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:40.808723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 845/1708] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:46:40 2022] GET /identity => generated 270 bytes in 44 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:40.810041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c1f63-f7bf-475f-90c0-ec4273b975cc None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:40.821423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4877a1b2-52e7-4063-add7-89f58ebc2999 None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:40.822973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4877a1b2-52e7-4063-add7-89f58ebc2999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:40.942732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 864/1709] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:40.945270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c179bb-d060-4582-85cc-d6667d2aa916 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:40.949947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c179bb-d060-4582-85cc-d6667d2aa916 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:40.950423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c179bb-d060-4582-85cc-d6667d2aa916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.040740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:41.045874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f2c179bb-d060-4582-85cc-d6667d2aa916 None None] MFA Rules not processed for user `0b262e34779546fdb0c773cf81a81f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:41.050109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f2c179bb-d060-4582-85cc-d6667d2aa916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:41.148335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c1f63-f7bf-475f-90c0-ec4273b975cc None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:41.169893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:41.171910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.172594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.174079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] PATH_INFO: `/v3/projects/69085c87d3b54fb9acfb9305be4c23dd/users/077dee62d31f46b08e121239f3f81a4f/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.197982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 865/1710] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:41.199505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.199838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.200122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.235144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:41.237560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:41.243562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=69085c87d3b54fb9acfb9305be4c23dd, user_id=077dee62d31f46b08e121239f3f81a4f, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:41.257111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:41.313499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 846/1711] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:40 2022] PUT /identity/v3/projects/69085c87d3b54fb9acfb9305be4c23dd/users/077dee62d31f46b08e121239f3f81a4f/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:41.314925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:41.347207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:41.400719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 866/1712] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:41.404164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:41.407033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:41.463952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561b934-0ffd-4813-8ddd-776883fa986e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:41.600089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920d7a7-a9cf-43e6-acb0-2700da6c75fb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:41.619090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:41.620017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.620444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.621371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.658680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:41.659878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.659878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.659878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.680917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:41.682450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:41.735708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 867/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:41.736665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:41.863054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:41.867744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce79629-07c0-431d-a2c1-67750e3a1cf5 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:41.905363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 847/1714] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:41.907915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.912807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.913361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.934422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65d2e-5fe9-41a8-b7ba-8e9e4f7dac38 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:41.964879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:41.972622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:41.972993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:41.973325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:41.974124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:41.991276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:42.018311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] MFA Rules not processed for user `077dee62d31f46b08e121239f3f81a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:42.020250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:42.118515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 868/1715] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:41 2022] POST /identity/v3/projects => generated 389 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:42.121824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:42.123386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:42.224259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:42.291516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 848/1716] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:42.292843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:42.337468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7149-6d61-461e-a534-603a69af06d8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:42.342454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af1d15-0f6a-4a62-8cc1-3be442d55e3d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:42.350776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:42.352901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:42.353295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:42.353670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:42.354408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:42.356987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:42.363484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:42.364283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:42.364598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:42.364904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] PATH_INFO: `/v3/projects/c17410fa824a4e80be7ea19c30af99f8` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:42.370818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c17410fa824a4e80be7ea19c30af99f8)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:42.379939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:42.387191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:42.428763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 849/1717] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:42 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:42.440210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:42.493228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff934c-6c39-4607-a39f-8f89a0802023 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:42.536145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c17410fa824a4e80be7ea19c30af99f8', 'request_id': 'req-a7b69d5f-c730-4864-8732-10d08d948080'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:42.557642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c17410fa824a4e80be7ea19c30af99f8', 'request_id': 'req-a7b69d5f-c730-4864-8732-10d08d948080'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:42.573124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c17410fa824a4e80be7ea19c30af99f8', 'request_id': 'req-a7b69d5f-c730-4864-8732-10d08d948080'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:42.605780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] The token cache is being invalidate because project c17410fa824a4e80be7ea19c30af99f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:42.606318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:42.607861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7b69d5f-c730-4864-8732-10d08d948080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:42.614136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 869/1718] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:42 2022] DELETE /identity/v3/projects/c17410fa824a4e80be7ea19c30af99f8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:42.617182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0547e-44d8-4947-93db-653c494b89e3 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:42.617473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0547e-44d8-4947-93db-653c494b89e3 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:42.619920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0547e-44d8-4947-93db-653c494b89e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:42.654335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:42.655084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:42.656260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:42.658095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:42.658816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:42.664212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:42.679642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-38f0547e-44d8-4947-93db-653c494b89e3 None None] MFA Rules not processed for user `468522570ba7419c87da6d05fc0d1ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:42.683919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-38f0547e-44d8-4947-93db-653c494b89e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:42.757687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 850/1719] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:42 2022] POST /identity/v3/projects => generated 373 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:42.759536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:42.768361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:42.874184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 870/1720] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:42.874184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:42.874184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:42.874184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:42.917042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:42.920153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.057820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 871/1721] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:43.057820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.057820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.062181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f826e4-9e5b-4112-90ed-7fa84d77c4a9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.097404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.098049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.098336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.098614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.099275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.100973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.146266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 851/1722] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:43.147167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.158613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.192239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:43.220320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f6f13-8c44-4493-ba80-47658a32c0dc admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.237422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.239937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.241780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.242381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.243989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.251646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197512-a5a5-48ff-8e16-1e3d864a6253 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.270909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7114d41-b9d1-49bc-bec0-f25ba44f242f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.295317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.296598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.297160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.297503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.300288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.305800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 852/1723] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:43 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:43.310602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.314272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.314734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.390713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.392833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.421654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:43.423595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:43.460892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 872/1724] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:43.462076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.484336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 853/1725] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:43.484336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.556447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3330c-2ff6-45c8-887b-2fbd27d1c192 service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.569198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fa620-82a1-4ebb-829c-e56a1af88645 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.595438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.596766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.596766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.596766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] PATH_INFO: `/v3/projects/6613810df92d4a06856c47295e5aaa61/users/ed07dce4147b46a5bdc9a6700ec3ea6f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.606576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.607212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.607492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.607761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.608404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.611586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.656707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 854/1726] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:43.658211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.663755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6613810df92d4a06856c47295e5aaa61, user_id=ed07dce4147b46a5bdc9a6700ec3ea6f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.666505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.689784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd8df-59fd-4819-9805-7eff61eaaeeb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.700580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 873/1727] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:43 2022] PUT /identity/v3/projects/6613810df92d4a06856c47295e5aaa61/users/ed07dce4147b46a5bdc9a6700ec3ea6f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:43.701626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:43.734087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:43.934722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.936552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.939022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.939423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.940222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:43.942733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:43.949891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:43.965744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bce60-49f4-477f-9535-0c240dcef12f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:43.978788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:43.979735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:43.979982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:43.980165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] PATH_INFO: `/v3/projects/74f03994bff44a289d6b39a7301db04b/users/ebfd92e656df42d9b33e1ce7d6579d56/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:43.985565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 855/1728] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:43.985565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.015853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f03994bff44a289d6b39a7301db04b, user_id=ebfd92e656df42d9b33e1ce7d6579d56, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.019320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.019791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6758-0123-4a8c-8f11-94d06a0cba1c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.036625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.040801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.041388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.041778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.044827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.049515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.049978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 874/1729] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:43 2022] PUT /identity/v3/projects/74f03994bff44a289d6b39a7301db04b/users/ebfd92e656df42d9b33e1ce7d6579d56/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:44.051110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.086470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 856/1730] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:44.090821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.127073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faad4b0-3791-4087-9a6b-37e413548626 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.243696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac9d9b-1f39-4d15-8e3e-ddfffee936f9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.257328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.259350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.259647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.259947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.261970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.268670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.334417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.338638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.338947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.339217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.339924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.341578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46640b7-a979-40ef-9815-e3df4137d964 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.370254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 875/1731] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:44 2022] POST /identity/v3/projects => generated 363 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:44.370254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.379437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 857/1732] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:44.385147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.385551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.385792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.409819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:44.449022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] MFA Rules not processed for user `79acdbc2c11c4459928eab5b83a6755d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:44.453822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:44.618205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 858/1733] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:44.618205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.667637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953761f4-fa3c-4760-b85c-671ef98521cb admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.682440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.684319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.685986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.686841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] PATH_INFO: `/v3/projects/6613810df92d4a06856c47295e5aaa61/users/ed07dce4147b46a5bdc9a6700ec3ea6f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.691471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6613810df92d4a06856c47295e5aaa61, user_id=ed07dce4147b46a5bdc9a6700ec3ea6f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.703053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.718878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:44.740461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 876/1734] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:44 2022] PUT /identity/v3/projects/6613810df92d4a06856c47295e5aaa61/users/ed07dce4147b46a5bdc9a6700ec3ea6f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:44.742022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:44.767300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31873a11-df2b-467d-83ec-eff662ffd3e1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.918541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b999c7-4b1d-4d5f-a959-154bb557dd07 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:44.924647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:44.926211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.926557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.926911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] PATH_INFO: `/v3/projects/74f03994bff44a289d6b39a7301db04b/users/ebfd92e656df42d9b33e1ce7d6579d56/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:44.929161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f03994bff44a289d6b39a7301db04b, user_id=ebfd92e656df42d9b33e1ce7d6579d56, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:44.932437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8691b3a-d4e0-4d0b-8526-f4267c86b3cb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:44.949040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 877/1735] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:44 2022] PUT /identity/v3/projects/74f03994bff44a289d6b39a7301db04b/users/ebfd92e656df42d9b33e1ce7d6579d56/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:44.952068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebb200-98de-4598-979e-88359b42390e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:44.952846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebb200-98de-4598-979e-88359b42390e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:44.954998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebb200-98de-4598-979e-88359b42390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.028054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f4ebb200-98de-4598-979e-88359b42390e None None] MFA Rules not processed for user `ed07dce4147b46a5bdc9a6700ec3ea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:45.029581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4ebb200-98de-4598-979e-88359b42390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:45.040831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.043006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.045524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.058969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.059672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.061666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72baaf-daad-4583-8df5-5ed39ee4d9c4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.147145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 859/1736] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:44 2022] POST /identity/v3/users => generated 282 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:45.164561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.164744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.165821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.259890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 878/1737] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:45.261234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebb200-98de-4598-979e-88359b42390e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.283740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] MFA Rules not processed for user `ebfd92e656df42d9b33e1ce7d6579d56`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:45.285074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:45.310203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:45.372984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebb200-98de-4598-979e-88359b42390e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:45.383059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.384741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.385345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.385882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.387901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.390173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.403307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 879/1738] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:45.404824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.407546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:45.491391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f1fe-9239-4bc3-9a9f-c4e07235367a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:45.496876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.497999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.498510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.499017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.514040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.517139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.530001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 880/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:45.545437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.609084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496979c-d2b1-4e4d-a5aa-4f8289fac234 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:45.633777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.634421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.634702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.634973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.645325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.646530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.703707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 860/1740] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 547 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:45.704782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.714148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 881/1741] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:45.715110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.723091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:45.751484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfada64-4f4c-4141-9e96-495f4991d833 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:45.782523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a2d8-6c26-4432-91d9-2e0dbdb33b95 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:45.783994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.784889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.785036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.785287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] PATH_INFO: `/v3/projects/2639aa0e0f794a389f2f373e17c1c2f7/users/f48d1fe959614fb2aef715ef270636e6/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.793961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:45.794595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.794876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.801199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:45.804247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:45.830125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2639aa0e0f794a389f2f373e17c1c2f7, user_id=f48d1fe959614fb2aef715ef270636e6, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.832800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.876590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 861/1742] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:45 2022] PUT /identity/v3/projects/2639aa0e0f794a389f2f373e17c1c2f7/users/f48d1fe959614fb2aef715ef270636e6/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:45.877537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:45.966357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:45.976582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:45.979313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565ffba-2205-46ca-86fc-3e321faa97b2 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:45.992255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 882/1743] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:45.994052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:45.994775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:45.995571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.019302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e53b94-7df5-451d-8fbc-5203a9d71f64 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:46.033120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:46.033953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.034238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.034510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] PATH_INFO: `/v3/users/0b262e34779546fdb0c773cf81a81f1b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.036231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] MFA Rules not processed for user `f48d1fe959614fb2aef715ef270636e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:46.037244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b262e34779546fdb0c773cf81a81f1b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:46.038328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:46.042005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:46.134923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b262e34779546fdb0c773cf81a81f1b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:46.146022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b262e34779546fdb0c773cf81a81f1b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:46.147445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:46.176291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b262e34779546fdb0c773cf81a81f1b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:46.194322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b262e34779546fdb0c773cf81a81f1b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:46.197754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-816e8668-0591-4a42-9a31-d6fead85ec25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b262e34779546fdb0c773cf81a81f1b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:46.217668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 862/1744] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:45 2022] DELETE /identity/v3/users/0b262e34779546fdb0c773cf81a81f1b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:46.220490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.220779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.224627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.226087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 883/1745] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:46.232146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:46.256812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:46.258235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:46.262207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2bf0-1357-4d95-b9a8-123c98367391 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:46.383779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 863/1746] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:46.403333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:46.408558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:46.459591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:46.460320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.460663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.461087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.461792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:46.463804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:46.552443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 884/1747] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:46 2022] POST /identity/v3/projects => generated 365 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:46:46.555404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:46.560781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:46.657864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df83880-30fd-410e-9d6d-bdf764cb88c0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:46.722616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:46.815749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267cef2-ad8b-41a6-aa28-ca428b6338c1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:46.833735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:46.834374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.834687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.835301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.835953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:46.837978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:46.885936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 885/1748] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:46 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:46.890037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:46.914212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c167893-dd3b-4372-b7b3-bd071e6ae406 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:46.927861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:46.928500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.928787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.929110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.930242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:46.933225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:46.934639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:46.934843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:46.936437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:46.936982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:46.959886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:46.963785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:46.968619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 886/1749] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:46.972633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:46.994021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d90e2-9951-4e6b-9d0d-f64cead96735 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:47.001765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 864/1750] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:47.024026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:47.024559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:47.024848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:47.025161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] PATH_INFO: `/v3/projects/07fec00e2297439ba80f6212554deb66/users/7056caa4dc874d3a8bdee8925fbb987e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:47.081825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fec00e2297439ba80f6212554deb66, user_id=7056caa4dc874d3a8bdee8925fbb987e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:47.092941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:47.130261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 887/1751] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:46 2022] PUT /identity/v3/projects/07fec00e2297439ba80f6212554deb66/users/7056caa4dc874d3a8bdee8925fbb987e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:46:47.133161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:47.148066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c3dff-1b40-44d7-835f-f0fefcbc8571 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:47.355113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:47.355775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:47.356111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:47.356383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:47.357366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:47.364224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74849e8e-0f67-4622-93de-5c74ed48ba6c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:47.414845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 865/1752] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:47.418559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:47.439952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a1f4-0cf0-4513-8e0b-0aebf5d9c045 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:47.457630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:47.461065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:47.461365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:47.461673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] PATH_INFO: `/v3/projects/07fec00e2297439ba80f6212554deb66/users/7056caa4dc874d3a8bdee8925fbb987e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:47.465943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fec00e2297439ba80f6212554deb66, user_id=7056caa4dc874d3a8bdee8925fbb987e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:47.468606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:47.497784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 888/1753] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:47 2022] PUT /identity/v3/projects/07fec00e2297439ba80f6212554deb66/users/7056caa4dc874d3a8bdee8925fbb987e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:47.509738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:47.510285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:47.510861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:47.566910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] MFA Rules not processed for user `7056caa4dc874d3a8bdee8925fbb987e`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:47.568768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:47.697129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 866/1754] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:47.710703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:47.713330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:47.732166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:48.003426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd58b-476e-4ba7-8b8e-db575a448834 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:48.018496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:48.019269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:48.019570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:48.026868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:48.064864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:48.066551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:48.104770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 889/1755] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:48.256426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:48.258651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:48.416030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:48.430260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c995c-7aad-4e42-a1b0-44fba6bea01e service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:48.435526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:48.436452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:48.437012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:48.437434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:48.443188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:48.452986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:48.515397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbbb12-f3d5-47ec-aa1e-b237b8434f5b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:48.522935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:48.544119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:48.545380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:48.547063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] PATH_INFO: `/v3/projects/83f5ad9dfb304c5c903d8c1e9e81d63f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:48.554270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f5ad9dfb304c5c903d8c1e9e81d63f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:48.561588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:48.602139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:48.609547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:48.646561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 890/1756] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:48.708290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f5ad9dfb304c5c903d8c1e9e81d63f', 'request_id': 'req-7a230f3a-375f-4a65-9832-ccb34b9c1750'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:48.719631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f5ad9dfb304c5c903d8c1e9e81d63f', 'request_id': 'req-7a230f3a-375f-4a65-9832-ccb34b9c1750'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:48.732395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f5ad9dfb304c5c903d8c1e9e81d63f', 'request_id': 'req-7a230f3a-375f-4a65-9832-ccb34b9c1750'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:48.771960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] The token cache is being invalidate because project 83f5ad9dfb304c5c903d8c1e9e81d63f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:48.773259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:48.774769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7a230f3a-375f-4a65-9832-ccb34b9c1750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:48.781246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 867/1757] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:48 2022] DELETE /identity/v3/projects/83f5ad9dfb304c5c903d8c1e9e81d63f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:48.785211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:48.801545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020efc55-8b22-46e5-9491-75c97815db13 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:48.827825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2555a-ff09-4fda-bf01-458ac6c5c725 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:48.828115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2555a-ff09-4fda-bf01-458ac6c5c725 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:48.828436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2555a-ff09-4fda-bf01-458ac6c5c725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:48.869696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6ca2555a-ff09-4fda-bf01-458ac6c5c725 None None] MFA Rules not processed for user `a0101b6f23d04016a89c986621ba5dde`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:48.872680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6ca2555a-ff09-4fda-bf01-458ac6c5c725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:48.953976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:48.955024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:48.955412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:48.955815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] PATH_INFO: `/v3/users/ed07dce4147b46a5bdc9a6700ec3ea6f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:48.957306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed07dce4147b46a5bdc9a6700ec3ea6f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:48.959511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:48.995482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 868/1758] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:49.098243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:49.100717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed07dce4147b46a5bdc9a6700ec3ea6f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:49.102058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed07dce4147b46a5bdc9a6700ec3ea6f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:49.131782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed07dce4147b46a5bdc9a6700ec3ea6f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:49.143873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed07dce4147b46a5bdc9a6700ec3ea6f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:49.153929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed07dce4147b46a5bdc9a6700ec3ea6f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:49.167152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 891/1759] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:48 2022] DELETE /identity/v3/users/ed07dce4147b46a5bdc9a6700ec3ea6f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:49.543954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a3cc-4cdc-4581-8dd0-092738d68eb3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:49.544463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a3cc-4cdc-4581-8dd0-092738d68eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:49.544541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a3cc-4cdc-4581-8dd0-092738d68eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:49.567542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-86f3a3cc-4cdc-4581-8dd0-092738d68eb3 None None] MFA Rules not processed for user `0fca332203b9473fa8b07286bc44300f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:49.568753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-86f3a3cc-4cdc-4581-8dd0-092738d68eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:49.643419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 869/1760] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:49.793169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:49.801034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:49.977165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb148b51-4547-41e7-acfe-061222372616 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:49.986314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:49.987391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:49.987778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:49.988154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] PATH_INFO: `/v3/projects/6613810df92d4a06856c47295e5aaa61` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:49.989580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6613810df92d4a06856c47295e5aaa61)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:49.992054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.079860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6613810df92d4a06856c47295e5aaa61', 'request_id': 'req-00e9e1b8-903d-4618-a480-4519dea2051b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:50.087255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6613810df92d4a06856c47295e5aaa61', 'request_id': 'req-00e9e1b8-903d-4618-a480-4519dea2051b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:50.102544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6613810df92d4a06856c47295e5aaa61', 'request_id': 'req-00e9e1b8-903d-4618-a480-4519dea2051b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:50.144415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] The token cache is being invalidate because project 6613810df92d4a06856c47295e5aaa61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:50.144719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:50.145445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00e9e1b8-903d-4618-a480-4519dea2051b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:50.147454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 892/1761] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:49 2022] DELETE /identity/v3/projects/6613810df92d4a06856c47295e5aaa61 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:50.152513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.152868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.153177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.175382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:50.176605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:50.200577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:50.338156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 870/1762] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:50.346712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.347000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.347283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.408589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:50.410311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:50.434062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 893/1763] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:50.438554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.451617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7c349-d906-4028-8dcb-41cd2e6ad201 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.475947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.476494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.476799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.480333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.481162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:50.481490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.483373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.514179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 871/1764] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:46:50.517745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.536289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f578c9f-7f99-4a52-b0db-93d04c2a054b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.551549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.553734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.554157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.554999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.556997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.562516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.617807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 894/1765] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:50 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:50.627152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.637579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72b82d-e193-486d-a130-8d50f26eb5ab admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.739014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.739716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.740137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.740485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.741333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.743623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.772833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 872/1766] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:50 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:50.780452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.806911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7afdcd-e8f9-4fae-9e54-8716cc94f575 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.820798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.822570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.824080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.824675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.825451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.827356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.840758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 895/1767] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:50.845149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.866370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a84f3-c427-475e-93c4-9f4a935a2795 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.881509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.882183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.882536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.889537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.889537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.889537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab11d8-d7b1-4d5b-a076-15799b95bc28 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:50.928528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 873/1768] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:50.933361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:50.951357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691af5e6-c1c1-4f31-ae0e-88d82b3fe176 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:50.957079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:50.958143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:50.958514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:50.958767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] PATH_INFO: `/v3/projects/1e336f26cef1489cb7e8c100bdc012f0/users/18d8011f025248fcabe33532252176db/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:50.972948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:50.979249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e336f26cef1489cb7e8c100bdc012f0, user_id=18d8011f025248fcabe33532252176db, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:50.982354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.008997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 896/1769] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:50 2022] PUT /identity/v3/projects/1e336f26cef1489cb7e8c100bdc012f0/users/18d8011f025248fcabe33532252176db/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:51.012478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.012951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.013400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.062544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] MFA Rules not processed for user `18d8011f025248fcabe33532252176db`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:51.066182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:51.176659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 874/1770] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:51.180590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.348563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5d82c-8bcd-4783-95be-80c423b7a25d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.363834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.364563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.364848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.365252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.365986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.368358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.408880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 897/1771] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:51 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:46:51.413235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.425919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537c9ff-5228-4edb-bbfc-40ffaf703e9f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.435602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:51.487002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.487624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.487909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.488186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.488817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.490757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.515276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 875/1772] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:51 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:51.517047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.527398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecf0-4379-4735-9f2b-3ce9639710b7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.532626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.533296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.533573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.533846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.534488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.536068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.547394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 898/1773] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:51.551675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.562692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad0573-e56d-4e55-a062-3a83e1a99476 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.567959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.568639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.568921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.569233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] PATH_INFO: `/v3/projects/a1b708801f624f108c72bd534a0114be/users/70a5f4ba64b34459a05180aae8f0d4e7/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.586677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b708801f624f108c72bd534a0114be, user_id=70a5f4ba64b34459a05180aae8f0d4e7, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.590576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.607484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 876/1774] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:51 2022] PUT /identity/v3/projects/a1b708801f624f108c72bd534a0114be/users/70a5f4ba64b34459a05180aae8f0d4e7/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:51.616767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.629672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5459-bc7f-4445-ad18-40517dcb8f78 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.700126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.700816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.701198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.701544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.702316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.705031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e8689-afce-4d84-9cf0-28fce8bbb54b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.722858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 899/1775] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:51.725849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.739724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0a1b3-6f47-4887-b291-54bec71228e2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:51.745648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:51.746460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.746748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.747022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] PATH_INFO: `/v3/projects/a1b708801f624f108c72bd534a0114be/users/70a5f4ba64b34459a05180aae8f0d4e7/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.748930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b708801f624f108c72bd534a0114be, user_id=70a5f4ba64b34459a05180aae8f0d4e7, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:51.751632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:51.762458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 877/1776] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:51 2022] PUT /identity/v3/projects/a1b708801f624f108c72bd534a0114be/users/70a5f4ba64b34459a05180aae8f0d4e7/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:46:51.774245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:51.774542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:51.774814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:51.810910 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] MFA Rules not processed for user `70a5f4ba64b34459a05180aae8f0d4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:51.812196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:51.876077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:51.908621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 900/1777] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:51.921736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:51.923629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:52.011269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75495-d4ff-4024-aad7-b0249b21e497 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:52.015619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:52.017060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.017415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.017765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.035549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:52.040417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a690bd2-419a-45f6-b4f9-e68f28cf7bfc service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:52.068354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 878/1778] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:52.611488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:52.683260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b84ec-d75b-41cc-bdcd-89bf4d45facc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:52.687793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:52.688468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.688754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.689072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] PATH_INFO: `/v3/users/18d8011f025248fcabe33532252176db` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.690202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d8011f025248fcabe33532252176db)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:52.692119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:52.741684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d8011f025248fcabe33532252176db'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.742341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d8011f025248fcabe33532252176db'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.756648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d8011f025248fcabe33532252176db'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.763980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d8011f025248fcabe33532252176db'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.772617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d8011f025248fcabe33532252176db'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.780029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 901/1779] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:52 2022] DELETE /identity/v3/users/18d8011f025248fcabe33532252176db => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:52.785318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.786164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.786516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.794723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:52.796019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:52.812506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:52.813770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:52.868322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f445-873d-4f63-9722-667e68fcc088 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:52.873444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:52.874192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.874481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.874759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] PATH_INFO: `/v3/users/f48d1fe959614fb2aef715ef270636e6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.876105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48d1fe959614fb2aef715ef270636e6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:52.878194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:52.883187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:52.888264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 879/1780] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:52.899317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:52.900645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:52.912714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:52.940399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48d1fe959614fb2aef715ef270636e6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.941100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48d1fe959614fb2aef715ef270636e6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.955170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48d1fe959614fb2aef715ef270636e6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.962373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48d1fe959614fb2aef715ef270636e6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.967592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83960f-ce61-4039-acdf-a3d072fa3796 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:52.971061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-546217f5-cb5d-436c-8c32-3377ebe88067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48d1fe959614fb2aef715ef270636e6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:52.971549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:52.972162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.972439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.972707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.974509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:52.977381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 902/1781] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:52 2022] DELETE /identity/v3/users/f48d1fe959614fb2aef715ef270636e6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:52.982095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:52.982558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:52.982841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:52.986542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:52.987940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:52.997719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 880/1782] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:46:53.002509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:53.003692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.065753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 903/1783] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:46:53.079385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:53.080896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.092259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b422-5d1b-42f9-8746-01de446352ef service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:53.145578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:53.146211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:53.146493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:53.146769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:53.158145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:53.159720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:53.176591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 881/1784] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:53.512262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:53.513787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.527965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbaa42-1bae-4ad9-b435-bd6ba9bd0b46 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:53.535405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:53.536468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:53.536949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:53.537460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] PATH_INFO: `/v3/projects/1e336f26cef1489cb7e8c100bdc012f0` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:53.538815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e336f26cef1489cb7e8c100bdc012f0)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:53.542404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:53.624224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e336f26cef1489cb7e8c100bdc012f0', 'request_id': 'req-33572859-32ff-4fb6-983e-d00d25fc22c5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.634307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e336f26cef1489cb7e8c100bdc012f0', 'request_id': 'req-33572859-32ff-4fb6-983e-d00d25fc22c5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.643190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e336f26cef1489cb7e8c100bdc012f0', 'request_id': 'req-33572859-32ff-4fb6-983e-d00d25fc22c5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.661371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] The token cache is being invalidate because project 1e336f26cef1489cb7e8c100bdc012f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:53.661708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.663777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.665593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 904/1785] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:53 2022] DELETE /identity/v3/projects/1e336f26cef1489cb7e8c100bdc012f0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:53.670438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:53.682146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7705-d4a5-4151-b802-b8643b72ab69 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:53.725882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:53.727355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.739541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:53.785258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:53.785909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:53.786447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:53.786447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] PATH_INFO: `/v3/users/70a5f4ba64b34459a05180aae8f0d4e7` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:53.787620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a5f4ba64b34459a05180aae8f0d4e7)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:53.789622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:53.807890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572859-32ff-4fb6-983e-d00d25fc22c5 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:53.812016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:53.812712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:53.813647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:53.814352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:53.816348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.847069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a5f4ba64b34459a05180aae8f0d4e7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.858543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a5f4ba64b34459a05180aae8f0d4e7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.861378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:53.883993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a5f4ba64b34459a05180aae8f0d4e7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.893908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a5f4ba64b34459a05180aae8f0d4e7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.894896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a5f4ba64b34459a05180aae8f0d4e7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:53.903371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 882/1786] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:53 2022] DELETE /identity/v3/users/70a5f4ba64b34459a05180aae8f0d4e7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:53.904784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:53.905629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:53.906488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:53.907285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:53.918947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 905/1787] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:54.017010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660702-8542-4919-8ae1-fd7d7be08cbb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.024054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.024946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.025359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.025738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] PATH_INFO: `/v3/projects/2639aa0e0f794a389f2f373e17c1c2f7` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.027274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2639aa0e0f794a389f2f373e17c1c2f7)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.029770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.105882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2639aa0e0f794a389f2f373e17c1c2f7', 'request_id': 'req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.116033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2639aa0e0f794a389f2f373e17c1c2f7', 'request_id': 'req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.124989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2639aa0e0f794a389f2f373e17c1c2f7', 'request_id': 'req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.139693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] The token cache is being invalidate because project 2639aa0e0f794a389f2f373e17c1c2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:54.140483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.141550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.145002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 883/1788] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:53 2022] DELETE /identity/v3/projects/2639aa0e0f794a389f2f373e17c1c2f7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:54.147980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.150850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:54.172143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1d68-4b87-4cc8-826a-4a1eb91f530b service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.265628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.269050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:54.285781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.286802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.287095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.287416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] PATH_INFO: `/v3/users/7056caa4dc874d3a8bdee8925fbb987e` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.289618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7056caa4dc874d3a8bdee8925fbb987e)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.301622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.382137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7056caa4dc874d3a8bdee8925fbb987e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.382956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7056caa4dc874d3a8bdee8925fbb987e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.395604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:54.400780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7056caa4dc874d3a8bdee8925fbb987e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.420841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7056caa4dc874d3a8bdee8925fbb987e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.436498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1ad2-7471-4fe2-9356-3f8c839b0644 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.436625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-10b81fb4-1e68-4ee8-bb8b-b4d71d565214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7056caa4dc874d3a8bdee8925fbb987e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.440887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.441718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.442108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.442446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] PATH_INFO: `/v3/projects/a1b708801f624f108c72bd534a0114be` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.444707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b708801f624f108c72bd534a0114be)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.444848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 906/1789] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:46:54 2022] DELETE /identity/v3/users/7056caa4dc874d3a8bdee8925fbb987e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:54.446783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.455566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:54.507057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b708801f624f108c72bd534a0114be', 'request_id': 'req-62e5b5d3-28f7-4774-a053-51c292978b2c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.517716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b708801f624f108c72bd534a0114be', 'request_id': 'req-62e5b5d3-28f7-4774-a053-51c292978b2c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.526224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b708801f624f108c72bd534a0114be', 'request_id': 'req-62e5b5d3-28f7-4774-a053-51c292978b2c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.539911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] The token cache is being invalidate because project a1b708801f624f108c72bd534a0114be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:54.540434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.541208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-62e5b5d3-28f7-4774-a053-51c292978b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.542800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 884/1790] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:54 2022] DELETE /identity/v3/projects/a1b708801f624f108c72bd534a0114be => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:54.547785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.548473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.548758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.570757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:54.571993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:54.636208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 907/1791] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:54.645934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.646218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.646518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.672093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:54.673606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:54.681745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 885/1792] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:54.686414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.702244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579283b-9189-4bd7-9cbf-6972a1d3984b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.707679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.708302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.708586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.708862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.709907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:54.710868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.712572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.729705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 908/1793] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:46:54.730763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.742766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd432a8-2a16-4919-9253-5afc8714d336 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.748562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.749377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.749665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.749963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.751673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.752592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.787702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 886/1794] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:54 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:54.796052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.805315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18918-154c-4614-843f-461381a7fbf8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.813530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.815776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:54.890398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafa3a4-d832-42ed-8836-5782afb3394d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.897187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.898373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.898927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.899427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] PATH_INFO: `/v3/projects/07fec00e2297439ba80f6212554deb66` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.900618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fec00e2297439ba80f6212554deb66)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.902660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.907450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:54.908064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:54.908346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:54.908621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:54.909285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:54.911206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:54.934730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 909/1795] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:54 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:54.938971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:54.948948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f9aa1-26a3-4576-b7a1-217a96b8fbd9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:54.958482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fec00e2297439ba80f6212554deb66', 'request_id': 'req-a49ef392-9852-4d01-b2f7-d32a40d599e6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.967611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fec00e2297439ba80f6212554deb66', 'request_id': 'req-a49ef392-9852-4d01-b2f7-d32a40d599e6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.975776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fec00e2297439ba80f6212554deb66', 'request_id': 'req-a49ef392-9852-4d01-b2f7-d32a40d599e6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.981026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:54.990903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] The token cache is being invalidate because project 07fec00e2297439ba80f6212554deb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:46:54.991383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.992482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a49ef392-9852-4d01-b2f7-d32a40d599e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:46:54.994891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 887/1796] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:46:54 2022] DELETE /identity/v3/projects/07fec00e2297439ba80f6212554deb66 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:55.002823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.003116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.003569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.008036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:55.019896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.020565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.020853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.021160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.021811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.023430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7d963-e434-40a2-9c96-96cc761fabf4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.024122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:55.025352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.035066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 888/1797] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:55.036249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 910/1798] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:46:55.040696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.042291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.042579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.042868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.043270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.054487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfef64-b97c-4101-8e13-268660a40ec4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.061279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.061930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.062213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.062486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.063425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.063875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:55.065035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.065175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.073395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 911/1799] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:55.077586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.082273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 889/1800] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:46:55.086633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.087657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450cfda-15f9-4c52-86f9-b15e9ac071e4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.092777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.093416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.093703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.093971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.095267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:46:55.095561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.097199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.100003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ddc68-d08e-48a6-b9ba-09abc5cd6085 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.104817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.105579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.105935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.106239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] PATH_INFO: `/v3/projects/31343832d13a43609368780f88bc67aa/users/19373743f3c1440eb2350e9ecc50fd87/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.111701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 912/1801] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:46:55.113768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.123505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8055e-0a8b-4c23-8bad-6b4a5d4b3a9d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.123664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31343832d13a43609368780f88bc67aa, user_id=19373743f3c1440eb2350e9ecc50fd87, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.126577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bd3ea-47af-4f6d-8017-10bc13dcd439 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.128248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.128847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.129161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.129441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.130057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.131931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.141494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 890/1802] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:55 2022] PUT /identity/v3/projects/31343832d13a43609368780f88bc67aa/users/19373743f3c1440eb2350e9ecc50fd87/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:55.154659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.154922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.155193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.164911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 913/1803] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:46:55.169021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.185244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] MFA Rules not processed for user `19373743f3c1440eb2350e9ecc50fd87`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:55.186372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.240268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fcb9-1d86-4cd0-be09-a05884e2b10d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.245263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.245882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.246161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.247148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.247921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.250101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.268625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 891/1804] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:55.278518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 914/1805] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:46:55.283004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.283634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.285334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.302932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a45fe6-1eec-414e-93a2-5931a514f6e5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.307957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.308581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.308858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.309289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.310067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.311740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.322658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 892/1806] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:55.327561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.339741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af8fe-4bc9-422e-aa3f-9059f02c0836 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.346302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.346926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.347213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.347492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.348522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.350542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.353882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22117431-5435-457b-ae97-c350ffbab0b3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.358144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.358925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.359203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.359474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.363809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 893/1807] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:46:55.367642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.371479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.373274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce4d06-05bf-423b-a2c0-7197e13400f5 service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.383594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 915/1808] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:46:55.387956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f5a1a-edec-4eb8-9828-52cdd712184e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.394038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.394718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.395006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.395290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] PATH_INFO: `/v3/projects/d4f1ad062764480a9eaa4e4a20ed9e36/users/8730daae55b54afe8d31934e41d94edf/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.412477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f1ad062764480a9eaa4e4a20ed9e36, user_id=8730daae55b54afe8d31934e41d94edf, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.415635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.431069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 894/1809] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:55 2022] PUT /identity/v3/projects/d4f1ad062764480a9eaa4e4a20ed9e36/users/8730daae55b54afe8d31934e41d94edf/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:55.441467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.441801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.442103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.484598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] MFA Rules not processed for user `8730daae55b54afe8d31934e41d94edf`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:55.486664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:55.520113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:55.577173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 916/1810] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:55.578304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.645258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:55.649645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe938ff-f787-4b75-9209-8b64f30604d3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.668005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.668788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.669171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.669523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.670272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.672214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.701597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 895/1811] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:55.706168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.716798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f9d4b-018a-48f7-81e1-f75111fd2d24 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.793428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.794048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.794483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.794614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.795331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.797502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.821560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 917/1812] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:55.825623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.838823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2147a1d-416a-4927-8895-ab700552d3dd admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.845524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.846156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.846414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.846693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.847318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.848895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.861759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 896/1813] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:55.866092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.881385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96dab4-6615-4c80-a9d5-4bbb08d5c0f8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.886351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.886978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.887263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.887539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.888176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.890181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abb5b-b9f5-4080-8cf6-0d1ddbff65c4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.904527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 918/1814] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:55.910111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:55.922950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb64c7-e433-4258-941c-63edb2bb1343 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:55.928378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:55.929229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.929572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.929910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] PATH_INFO: `/v3/projects/ce28efe45e6346baa687a6e226a5a0d5/users/31c2eeacefca463baab459f46f4dc41f/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:55.960279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce28efe45e6346baa687a6e226a5a0d5, user_id=31c2eeacefca463baab459f46f4dc41f, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:55.963296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:55.979062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 897/1815] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:55 2022] PUT /identity/v3/projects/ce28efe45e6346baa687a6e226a5a0d5/users/31c2eeacefca463baab459f46f4dc41f/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:55.990990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:55.991471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:55.991862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.020528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] MFA Rules not processed for user `31c2eeacefca463baab459f46f4dc41f`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:56.021771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:56.090430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 919/1816] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:56.095266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.155927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2f11-51af-4ff0-97a2-1e3caac69d9d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.160647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.161283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.161570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.161844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.162458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.164370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.190024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 898/1817] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:46:56 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:56.194735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.206516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f59493-944c-417e-a11b-b19f14a20486 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.238650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:56.262351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.262958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.263241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.263510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.264145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.266688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.293616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 920/1818] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:46:56 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:46:56.298262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.308800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea7dd-5db5-4e7b-8e25-ac70ab6b902b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.313420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.314018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.314303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.314577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.315210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.316772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.327392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 899/1819] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:56.331953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.341947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019652af-3b0a-45d5-a5e6-dd0c8b5f20ec admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.347239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.347897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.348181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.348488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] PATH_INFO: `/v3/projects/615731ee68864c45b10f4a9d5c13cf7f/users/75957105ef21482ea131021d9f919667/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.363868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615731ee68864c45b10f4a9d5c13cf7f, user_id=75957105ef21482ea131021d9f919667, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.366597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.380884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 921/1820] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:56 2022] PUT /identity/v3/projects/615731ee68864c45b10f4a9d5c13cf7f/users/75957105ef21482ea131021d9f919667/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:56.384971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.395616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6265847-43b2-4c43-8994-32c8ce364b7d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.451465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.452079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.452361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.452634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.453299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.455353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1b38c-b125-4f57-84b5-85de91745105 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.470048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 900/1821] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:46:56.472315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.483792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9ff6f-0d64-4625-ae92-0fe1238601d9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.490388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.491368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.491479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.491719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] PATH_INFO: `/v3/projects/615731ee68864c45b10f4a9d5c13cf7f/users/75957105ef21482ea131021d9f919667/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.493658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615731ee68864c45b10f4a9d5c13cf7f, user_id=75957105ef21482ea131021d9f919667, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.496487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.510057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 922/1822] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:46:56 2022] PUT /identity/v3/projects/615731ee68864c45b10f4a9d5c13cf7f/users/75957105ef21482ea131021d9f919667/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:46:56.520808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.521232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.521608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.551515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] MFA Rules not processed for user `75957105ef21482ea131021d9f919667`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:56.553325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:56.560094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:56.636767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 901/1823] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:46:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:56.648690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:56.650383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:56.710327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5919ecd-cd48-467d-b9df-781b2103f017 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:56.713996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:56.714684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:56.715039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:56.715405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:56.725596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:56.727079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c0ba-b0ef-462a-9884-a0a14403f512 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:56.737504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 923/1824] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:57.261490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:57.271348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5369-872f-4a98-90da-210e4bfe8761 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:57.274883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:57.275481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:57.275790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:57.276089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:57.333366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:57.334772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760e789-f2d3-4aaa-88d9-b975e6de1de7 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:57.343530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 902/1825] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:01.729502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:01.731032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:01.732557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:01.734249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 924/1826] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:47:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:01.747007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:01.747580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:01.747791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:01.837128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:01.838468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:02.003917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 903/1827] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:47:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:02.021257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:02.023559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:02.075313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe2a14-7385-4b50-897e-7357b0c07ea8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:02.085107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:02.090496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:02.090496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:02.090496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:02.102286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:02.103963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd103c1-9855-4d11-9d27-1f838d669279 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:02.119488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 925/1828] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:09.175168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:09.188133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb6611-397f-4c1a-a474-74bae16c058e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:09.197275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:09.198013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:09.198391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:09.198774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:09.239388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:09.283888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:09.286088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f42c4-00ec-47a7-aaa7-919c197b9e62 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:09.296656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 904/1829] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:09.940103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f9ad-80c5-4372-b15d-80396cff37df None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:09.941270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f9ad-80c5-4372-b15d-80396cff37df None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:09.941944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f9ad-80c5-4372-b15d-80396cff37df None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:09.943806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 926/1830] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:47:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:09.978744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:09.979016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:09.980358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:10.034032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:10.035272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:10.052695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 905/1831] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:10.071897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20f9ad-80c5-4372-b15d-80396cff37df None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:10.109754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20f9ad-80c5-4372-b15d-80396cff37df None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:10.118990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:10.120919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:10.121230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:10.121496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:10.140020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:10.142781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08575aae-06ce-4d9a-8e07-06c9b223f909 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:10.183325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 927/1832] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:20.011642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=16 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:20.166749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:20.448708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:20.512594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497c31f-6099-46d9-a1d6-dfef0267e194 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:20.517400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:20.518212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:20.518492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:20.518829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] PATH_INFO: `/v3/users/8730daae55b54afe8d31934e41d94edf` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:20.519954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8730daae55b54afe8d31934e41d94edf)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:20.521907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:20.570426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8730daae55b54afe8d31934e41d94edf'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:20.578244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8730daae55b54afe8d31934e41d94edf'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:20.595099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8730daae55b54afe8d31934e41d94edf'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:20.605198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8730daae55b54afe8d31934e41d94edf'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:20.605945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8730daae55b54afe8d31934e41d94edf'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:20.612804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 906/1833] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:20 2022] DELETE /identity/v3/users/8730daae55b54afe8d31934e41d94edf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:20.617051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:20.617384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:20.617683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:20.636724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:20.638025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:20.710401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 928/1834] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:20.720403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:20.721841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:20.796980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302c7a7-ce5d-4640-9834-14a93fe7f8a3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:20.800702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:20.801580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:20.801870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:20.802153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:20.811665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:20.813161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:20.823634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 907/1835] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:21.214706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:21.216204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:21.238826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d20aeb-8a45-40ac-a4f9-39f31e4ee63d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:21.253673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:21.255068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:21.255524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:21.256646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] PATH_INFO: `/v3/projects/d4f1ad062764480a9eaa4e4a20ed9e36` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:21.262275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f1ad062764480a9eaa4e4a20ed9e36)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:21.265848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:21.328485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f1ad062764480a9eaa4e4a20ed9e36', 'request_id': 'req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.335684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f1ad062764480a9eaa4e4a20ed9e36', 'request_id': 'req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.343463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f1ad062764480a9eaa4e4a20ed9e36', 'request_id': 'req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.356889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] The token cache is being invalidate because project d4f1ad062764480a9eaa4e4a20ed9e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:21.357313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.357973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.360318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 929/1836] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:21 2022] DELETE /identity/v3/projects/d4f1ad062764480a9eaa4e4a20ed9e36 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:21.364334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:21.374000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4174ab-ef0f-4d92-97f5-94af593a33d0 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:21.407874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:21.493047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:21.494003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:21.494389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:21.494871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] PATH_INFO: `/v3/users/31c2eeacefca463baab459f46f4dc41f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:21.496438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c2eeacefca463baab459f46f4dc41f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:21.498911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:21.584610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c2eeacefca463baab459f46f4dc41f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.599967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c2eeacefca463baab459f46f4dc41f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.616538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c2eeacefca463baab459f46f4dc41f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.624406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c2eeacefca463baab459f46f4dc41f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.625336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c2eeacefca463baab459f46f4dc41f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:21.632086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 908/1837] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:21 2022] DELETE /identity/v3/users/31c2eeacefca463baab459f46f4dc41f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:22.160981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:22.162527 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:22.178591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cb0c-0e58-4741-9f7a-4b1609d093e0 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:22.237780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:22.238397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:22.238590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:22.238862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] PATH_INFO: `/v3/projects/ce28efe45e6346baa687a6e226a5a0d5` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:22.239966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce28efe45e6346baa687a6e226a5a0d5)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:22.241917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:22.273897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:22.291268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce28efe45e6346baa687a6e226a5a0d5', 'request_id': 'req-0022f0f6-6c59-4321-ba24-c18109b331d6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.297421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce28efe45e6346baa687a6e226a5a0d5', 'request_id': 'req-0022f0f6-6c59-4321-ba24-c18109b331d6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.303700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce28efe45e6346baa687a6e226a5a0d5', 'request_id': 'req-0022f0f6-6c59-4321-ba24-c18109b331d6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.315874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] The token cache is being invalidate because project ce28efe45e6346baa687a6e226a5a0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:22.316165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.316786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.318327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 930/1838] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:22 2022] DELETE /identity/v3/projects/ce28efe45e6346baa687a6e226a5a0d5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:22.322632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:22.324031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:22.336669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bd77f-a0a8-4cc1-92e6-ce0c8f15d4d8 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:22.393235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:22.393872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:22.394156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:22.394432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] PATH_INFO: `/v3/users/75957105ef21482ea131021d9f919667` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:22.395557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75957105ef21482ea131021d9f919667)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:22.397498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:22.441901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:22.445765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75957105ef21482ea131021d9f919667'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.454729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75957105ef21482ea131021d9f919667'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.467295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75957105ef21482ea131021d9f919667'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.474316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75957105ef21482ea131021d9f919667'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.474994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7916bcb1-a476-4911-bfff-053760bff1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75957105ef21482ea131021d9f919667'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:22.480865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 909/1839] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:22 2022] DELETE /identity/v3/users/75957105ef21482ea131021d9f919667 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:22.852294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:22.854282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:22.873794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022f0f6-6c59-4321-ba24-c18109b331d6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:22.938864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:22.939651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:22.940079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:22.940431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] PATH_INFO: `/v3/projects/615731ee68864c45b10f4a9d5c13cf7f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:22.943363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615731ee68864c45b10f4a9d5c13cf7f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:22.946027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.016233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615731ee68864c45b10f4a9d5c13cf7f', 'request_id': 'req-0efeaabe-411b-4e61-b48b-5c978fe72d91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:23.021694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615731ee68864c45b10f4a9d5c13cf7f', 'request_id': 'req-0efeaabe-411b-4e61-b48b-5c978fe72d91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:23.027986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615731ee68864c45b10f4a9d5c13cf7f', 'request_id': 'req-0efeaabe-411b-4e61-b48b-5c978fe72d91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:23.041450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] The token cache is being invalidate because project 615731ee68864c45b10f4a9d5c13cf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:23.041744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:23.042540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0efeaabe-411b-4e61-b48b-5c978fe72d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:23.043877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 931/1840] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:22 2022] DELETE /identity/v3/projects/615731ee68864c45b10f4a9d5c13cf7f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:23.050658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.050950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.051230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.071566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:23.072767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:23.135068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 910/1841] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:23.144506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.144794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.145119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.164877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:23.166103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:23.174386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 932/1842] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:23.178646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.188328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c5da3-b415-46f6-850e-43cd51bcb17c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.192834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.193472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.193752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.194288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.194928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:23.195230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.196825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.208899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 911/1843] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:23.213144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.222840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5f723-5501-436e-b555-3151093e5561 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.227316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.227917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.228199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.228474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.229147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.231038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.257716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 933/1844] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:23.261797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.271841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb38-1d1d-44a9-9378-0d339ab413c5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.326537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.327622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.327910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.328181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.328806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.330712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.351135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 912/1845] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:23.355147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.365151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bb11-6190-4e5b-bc96-f43670d3a2ad admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.369590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.370174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.370453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.370722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.371342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.372927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.383239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 934/1846] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:23.387108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.397703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa7fc-3781-46d1-8590-57267ea67701 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.402445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.403040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.403323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.403599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.404238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.405947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bcd5e-4867-4bae-bb0a-f8804e20d960 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.416460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 913/1847] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:23.420499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.427400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:23.431107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a577-60e5-43a5-a51c-6a41dbbde897 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.437432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.438258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.438686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.439019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] PATH_INFO: `/v3/projects/00d73413f4c74657807d792d6cd2d096/users/78909bb756464fd3a4de32c007b04173/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.453974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d73413f4c74657807d792d6cd2d096, user_id=78909bb756464fd3a4de32c007b04173, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.456623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.469002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 935/1848] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:23 2022] PUT /identity/v3/projects/00d73413f4c74657807d792d6cd2d096/users/78909bb756464fd3a4de32c007b04173/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:23.480013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.480303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.480582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.510234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] MFA Rules not processed for user `78909bb756464fd3a4de32c007b04173`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:23.511448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:23.523944 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:23.579468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 914/1849] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:23.584253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.645831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4705eb7-d974-469c-b611-ff7eae95dd3f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.650491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.651106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.651389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.651664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.652307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.654781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.684068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 936/1850] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:23.688542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.701859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bdac7-7661-4e64-b6b7-a09fecb11511 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.760599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.761236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.761515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.761787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.762411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.764295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.786755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 915/1851] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:23 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:23.790551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.801339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa4bc0-7e22-4fd8-becc-7ee7ed36955b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.805829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.806419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.806694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.806961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.807584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.809228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.819779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 937/1852] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:23.823880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.833828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14ec8-4684-455c-9df2-e68f6a25529d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.838454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.839108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.839391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.839669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] PATH_INFO: `/v3/projects/e17a4f75c49949ec8bee373df3b9da96/users/2718b6525d4247099d91f29b3e46c10a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.855429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17a4f75c49949ec8bee373df3b9da96, user_id=2718b6525d4247099d91f29b3e46c10a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.857897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.873814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 916/1853] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:23 2022] PUT /identity/v3/projects/e17a4f75c49949ec8bee373df3b9da96/users/2718b6525d4247099d91f29b3e46c10a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:23.877990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.888044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29966-de47-4205-853b-5ed1a83451ec admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.950436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.951052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.951331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.951601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.952259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.953889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48f4e7-8bca-4671-a9b8-65df2f3a6e4c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:23.963846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 938/1854] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:23.968639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:23.978752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312ffee-3764-4224-b08a-8cf35538fcb1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:23.983334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:23.984563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:23.984842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:23.985145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] PATH_INFO: `/v3/projects/e17a4f75c49949ec8bee373df3b9da96/users/2718b6525d4247099d91f29b3e46c10a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:23.987048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17a4f75c49949ec8bee373df3b9da96, user_id=2718b6525d4247099d91f29b3e46c10a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:23.989949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:24.003513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 917/1855] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:23 2022] PUT /identity/v3/projects/e17a4f75c49949ec8bee373df3b9da96/users/2718b6525d4247099d91f29b3e46c10a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:24.016894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:24.017214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:24.017498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:24.062028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] MFA Rules not processed for user `2718b6525d4247099d91f29b3e46c10a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:24.063302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:24.114005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:24.151405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 939/1856] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:24.163056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:24.164783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:24.232079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d6f1-26f5-4674-b880-1cc4837cf476 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:24.235902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:24.236515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:24.236797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:24.237112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:24.246446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:24.247855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:24.257667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 918/1857] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:24.358929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:24.368308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30f38-780e-4332-a24c-9a0cd77ad399 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:24.371782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:24.372378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:24.372655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:24.372924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:24.429996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:24.431443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e01abe-185b-4f6e-8f0f-76ba5cf7d984 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:24.440758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 940/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:32.285211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:32.297230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:32.348080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03fdc-b989-4fb9-a660-0dd8ef46c93a service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:32.352558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:32.353213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:32.353497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:32.353776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] PATH_INFO: `/v3/users/78909bb756464fd3a4de32c007b04173` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:32.354901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78909bb756464fd3a4de32c007b04173)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:32.356800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:32.401673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78909bb756464fd3a4de32c007b04173'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:32.409317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78909bb756464fd3a4de32c007b04173'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:32.420878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78909bb756464fd3a4de32c007b04173'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:32.428191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78909bb756464fd3a4de32c007b04173'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:32.428931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78909bb756464fd3a4de32c007b04173'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:32.435243 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 919/1859] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:32 2022] DELETE /identity/v3/users/78909bb756464fd3a4de32c007b04173 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:32.440045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:32.440476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:32.440476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:32.461941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:32.463282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:32.525312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 941/1860] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:32.535841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:32.537372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:32.598022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87132e2-4e35-4361-aa33-87e3e77dbc38 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:32.602495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:32.603053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:32.603365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:32.603642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:32.612804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:32.614272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:32.623793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 920/1861] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:33.358461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:33.362271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:33.430253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfd086-858a-4e5c-9aa9-88757a838766 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:33.442761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:33.444122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:33.444757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:33.445170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] PATH_INFO: `/v3/projects/00d73413f4c74657807d792d6cd2d096` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:33.450656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d73413f4c74657807d792d6cd2d096)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:33.455673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:33.640437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d73413f4c74657807d792d6cd2d096', 'request_id': 'req-ed10f5fc-8d3d-40bd-8395-007a45ee061f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:33.652058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d73413f4c74657807d792d6cd2d096', 'request_id': 'req-ed10f5fc-8d3d-40bd-8395-007a45ee061f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:33.669024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d73413f4c74657807d792d6cd2d096', 'request_id': 'req-ed10f5fc-8d3d-40bd-8395-007a45ee061f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:33.694959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:33.715178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] The token cache is being invalidate because project 00d73413f4c74657807d792d6cd2d096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:33.715880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:33.718471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:33.743810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 942/1862] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:33 2022] DELETE /identity/v3/projects/00d73413f4c74657807d792d6cd2d096 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:33.745375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:33.762704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd7207-f785-457e-acb0-185e1bf58eca service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:33.879230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:33.963871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:33.964701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:33.966495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:33.967004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] PATH_INFO: `/v3/users/2718b6525d4247099d91f29b3e46c10a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:33.973156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2718b6525d4247099d91f29b3e46c10a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:33.992053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:34.082721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2718b6525d4247099d91f29b3e46c10a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.097755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2718b6525d4247099d91f29b3e46c10a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.127262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2718b6525d4247099d91f29b3e46c10a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.139713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2718b6525d4247099d91f29b3e46c10a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.141976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-76166c6e-264c-41c3-a232-5fdf3f4a3721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2718b6525d4247099d91f29b3e46c10a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.151403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 921/1863] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:33 2022] DELETE /identity/v3/users/2718b6525d4247099d91f29b3e46c10a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:34.220871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:34.223035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.241407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10f5fc-8d3d-40bd-8395-007a45ee061f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:34.339428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:34.340196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.340424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.340932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] PATH_INFO: `/v3/users/077dee62d31f46b08e121239f3f81a4f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.342188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077dee62d31f46b08e121239f3f81a4f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:34.345143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:34.419628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077dee62d31f46b08e121239f3f81a4f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.420321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077dee62d31f46b08e121239f3f81a4f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.436550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077dee62d31f46b08e121239f3f81a4f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.445930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077dee62d31f46b08e121239f3f81a4f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.463096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077dee62d31f46b08e121239f3f81a4f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.478459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 943/1864] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:34 2022] DELETE /identity/v3/users/077dee62d31f46b08e121239f3f81a4f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:34.483314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.483865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.484160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.511770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:34.513946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.614912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 922/1865] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:34.626327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:34.627977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.723055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:34.724550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.726381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d737a0-f9a6-4a8b-ab75-feb04ad20208 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:34.731765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:34.732444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.732729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.733057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.737714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc6ee-53e9-4909-9e35-3b9c883467e7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:34.743496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:34.743662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:34.744135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.744468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.744747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] PATH_INFO: `/v3/projects/e17a4f75c49949ec8bee373df3b9da96` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.745298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d5c45-9f04-4a5d-8ec1-88a15669101b service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:34.745892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17a4f75c49949ec8bee373df3b9da96)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:34.747804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:34.757197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 944/1866] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:34.803294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:34.806041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17a4f75c49949ec8bee373df3b9da96', 'request_id': 'req-3a0036e5-3222-45a9-a725-c82772f587aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.820302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17a4f75c49949ec8bee373df3b9da96', 'request_id': 'req-3a0036e5-3222-45a9-a725-c82772f587aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.830068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17a4f75c49949ec8bee373df3b9da96', 'request_id': 'req-3a0036e5-3222-45a9-a725-c82772f587aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.842591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] The token cache is being invalidate because project e17a4f75c49949ec8bee373df3b9da96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:34.842881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.843562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3a0036e5-3222-45a9-a725-c82772f587aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:34.845676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 923/1867] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:34 2022] DELETE /identity/v3/projects/e17a4f75c49949ec8bee373df3b9da96 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:34.850422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.850714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.850992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.872002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:34.873535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.888652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:34.942182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 945/1868] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:34.950561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:34.957118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:34.957485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:34.983062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:34.984290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:34.992641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 924/1869] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:34.996997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.018408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56086b9b-aacd-4720-b77a-d93bcd192c30 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.025458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.026289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.026733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.027164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.028166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:35.028741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.030757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.046835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 946/1870] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:35.051090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.060532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1378e54-8183-44df-a6a3-c0a368b11237 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.065803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.066409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.066804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.067252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.067955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.070153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.104496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 925/1871] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:35.105816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.117068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b4912-5a5d-42c5-a9fb-aefeaa329236 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.177263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.177883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.178307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.178594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.179289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.181258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.215368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 947/1872] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:35.220516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.233304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e11559-8d8b-415b-8141-e2c3a852e089 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.238589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.246909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.251828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.253905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.254403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.254903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.256026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.258726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.270268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0807e9-ce03-4986-9aa7-8cf352da5bea admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.284545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 926/1873] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:35.290065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.295844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.296254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.296425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] PATH_INFO: `/v3/projects/69085c87d3b54fb9acfb9305be4c23dd` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.299094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69085c87d3b54fb9acfb9305be4c23dd)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.301612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.304787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.330053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86a1a-a22a-4c78-a4ba-fb8be45a1308 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.339484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.340514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.341423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.341722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.342448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.344773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.360991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 927/1874] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:35.365845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.387792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a1e8a-c7b8-47d8-8dd7-1234b6947843 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.395281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69085c87d3b54fb9acfb9305be4c23dd', 'request_id': 'req-d75e61f7-9338-46f9-a1af-3efefdf7152b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.402675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69085c87d3b54fb9acfb9305be4c23dd', 'request_id': 'req-d75e61f7-9338-46f9-a1af-3efefdf7152b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.412134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69085c87d3b54fb9acfb9305be4c23dd', 'request_id': 'req-d75e61f7-9338-46f9-a1af-3efefdf7152b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.437347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] The token cache is being invalidate because project 69085c87d3b54fb9acfb9305be4c23dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:35.438970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.440397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.448526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 948/1875] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:35 2022] DELETE /identity/v3/projects/69085c87d3b54fb9acfb9305be4c23dd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:35.449621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.452121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.474756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.475458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.475806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.476040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] PATH_INFO: `/v3/projects/e36a303c592846d397857ef7fc796bc6/users/fb7852b66e6a4b44a62349887947e727/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.480662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:35.503520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36a303c592846d397857ef7fc796bc6, user_id=fb7852b66e6a4b44a62349887947e727, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.507151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d433ad0-8e84-4753-a511-82e37d12c1ea admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.538329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 928/1876] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:35 2022] PUT /identity/v3/projects/e36a303c592846d397857ef7fc796bc6/users/fb7852b66e6a4b44a62349887947e727/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:35.540463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.540888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.541246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.555557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e61f7-9338-46f9-a1af-3efefdf7152b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.584753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] MFA Rules not processed for user `fb7852b66e6a4b44a62349887947e727`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:35.586040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.656380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.657827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.658295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.658728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] PATH_INFO: `/v3/users/ebfd92e656df42d9b33e1ce7d6579d56` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.660185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebfd92e656df42d9b33e1ce7d6579d56)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.662422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.699498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:35.708741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 929/1877] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:35.713405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.716346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.735062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebfd92e656df42d9b33e1ce7d6579d56'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.736687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebfd92e656df42d9b33e1ce7d6579d56'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.738166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2514b6-2393-4367-aeca-be493fc887d5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.744609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.745609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.746136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.747365 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.748035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.750654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.751923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebfd92e656df42d9b33e1ce7d6579d56'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.760914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebfd92e656df42d9b33e1ce7d6579d56'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.769226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebfd92e656df42d9b33e1ce7d6579d56'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.775697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 949/1878] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:35 2022] DELETE /identity/v3/users/ebfd92e656df42d9b33e1ce7d6579d56 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:35.776272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.777678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.781601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 930/1879] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:35.785309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.787004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:35.837662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:35.857304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e7965-2f6a-41b8-bd1c-0b221bffd2be admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.863256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.863991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.864328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.864567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76ffd-3d1e-43d9-8507-8ffae98c3cc1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.865255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] PATH_INFO: `/v3/users/19373743f3c1440eb2350e9ecc50fd87` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.867023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=19373743f3c1440eb2350e9ecc50fd87)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.869075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.869374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.870108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.870397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.870677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.871940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.874021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.901018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 931/1880] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:35.903679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.914735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f98b9-77a9-4710-b35e-ec7431f20806 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.919876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.920561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.920989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.921287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.921965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.923715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.942204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 932/1881] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:35.947092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:35.951150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19373743f3c1440eb2350e9ecc50fd87'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.952037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19373743f3c1440eb2350e9ecc50fd87'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.958386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd90-fd61-47a7-87a3-2e347ef12979 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:35.963970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:35.964717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:35.965209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:35.965357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] PATH_INFO: `/v3/projects/1df110c21e7d4904b76e3bd30d05971b/users/a63e3bf614e2469b9f795e3c9580794e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:35.968487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19373743f3c1440eb2350e9ecc50fd87'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.977849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19373743f3c1440eb2350e9ecc50fd87'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.980790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df110c21e7d4904b76e3bd30d05971b, user_id=a63e3bf614e2469b9f795e3c9580794e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:35.983652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:35.987878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-19a2c0b3-3783-4bdd-bd5d-63320760f9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19373743f3c1440eb2350e9ecc50fd87'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:35.994247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 950/1882] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:35 2022] DELETE /identity/v3/users/19373743f3c1440eb2350e9ecc50fd87 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:36.000999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 933/1883] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:35 2022] PUT /identity/v3/projects/1df110c21e7d4904b76e3bd30d05971b/users/a63e3bf614e2469b9f795e3c9580794e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:36.001156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.001156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.001156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.002481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.004163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.021451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:36.023086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.103754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f62a9-bd3b-493f-83ce-f35716d4e719 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.106558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 951/1884] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:36.110323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.110323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.112939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.113824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.114838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.114838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.115510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.118719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.126139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3693a1c-caa3-46e2-bb36-34c25fe44797 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.131563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.132344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.132624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.132894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] PATH_INFO: `/v3/projects/74f03994bff44a289d6b39a7301db04b` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.134104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f03994bff44a289d6b39a7301db04b)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.137666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.138381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 934/1885] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:36.139437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.142203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.180114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:36.197280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f03994bff44a289d6b39a7301db04b', 'request_id': 'req-06d8ca67-a5f2-46bd-94db-29c0ec7db178'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.205395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f03994bff44a289d6b39a7301db04b', 'request_id': 'req-06d8ca67-a5f2-46bd-94db-29c0ec7db178'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.208150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:36.214157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f03994bff44a289d6b39a7301db04b', 'request_id': 'req-06d8ca67-a5f2-46bd-94db-29c0ec7db178'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.214853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79962-1bcf-4d6e-9617-1bdb2eca9c2a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.229989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] The token cache is being invalidate because project 74f03994bff44a289d6b39a7301db04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:36.230314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.231073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.232581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 952/1886] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:36 2022] DELETE /identity/v3/projects/74f03994bff44a289d6b39a7301db04b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:36.233392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.235253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.279256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.281625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.281625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.281625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.282244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.309357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca67-a5f2-46bd-94db-29c0ec7db178 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.314922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.315554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.315895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.316774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] PATH_INFO: `/v3/projects/1df110c21e7d4904b76e3bd30d05971b/users/a63e3bf614e2469b9f795e3c9580794e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.318519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df110c21e7d4904b76e3bd30d05971b, user_id=a63e3bf614e2469b9f795e3c9580794e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.321506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada3c51-410d-4280-951d-245295e94970 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.333040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 953/1887] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:36 2022] PUT /identity/v3/projects/1df110c21e7d4904b76e3bd30d05971b/users/a63e3bf614e2469b9f795e3c9580794e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:36.338830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00eb7-6be8-4106-9470-b6ac446264b2 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.339771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00eb7-6be8-4106-9470-b6ac446264b2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.340206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c00eb7-6be8-4106-9470-b6ac446264b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.354911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.356339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b4c58-b3ce-47a3-8bcd-9591759cf5b6 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.363435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-39c00eb7-6be8-4106-9470-b6ac446264b2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:36.364652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-39c00eb7-6be8-4106-9470-b6ac446264b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.367538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 935/1888] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:36.368857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.369204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.369505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.416200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] MFA Rules not processed for user `a63e3bf614e2469b9f795e3c9580794e`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:36.417607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.469666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 954/1889] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:36.475610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.475610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.475610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.494939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:36.496146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.506187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 955/1890] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:36.510545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.520443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f3559-d61f-452d-ae4b-6d0209f2a15d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.525803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.526427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.526753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.527089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.527947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:36.528261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.531766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.548009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 956/1891] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:36.548769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.553304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 936/1892] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:36.560130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c802d55-a63b-4dee-88e4-799eedc9b4a7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.564873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.566611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.587754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.588882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.589211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.589489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.590128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.592046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.600224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:36.663587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 957/1893] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:36.664641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.674029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533898-ab0e-4cc2-b403-64b812edccb4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.730262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:36.755503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6ff49-bf80-4587-935a-e928818fd429 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.760155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.760761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.761069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.761342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.762723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.763791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.764254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.764468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.764657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.764870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.790792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 958/1894] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:36.794682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.804613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209e6da-e580-4896-ac6e-3a92b4a965fc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.809454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.810180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.810470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.810735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.811370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.813045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.830256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 959/1895] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:36.834451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.845590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353401b8-b601-4b06-bb07-29b45e9cee6d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.849798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.850584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.851200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.851454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.851720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.851838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.852388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.854091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.861749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 937/1896] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:36.869829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 960/1897] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:36.871774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.873020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:36.874964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.885629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4917e19-7bb8-4bc3-b26b-7c6416a2996d service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.891897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.892592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.892869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.893187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] PATH_INFO: `/v3/projects/177434241f6a40a193bf3ff682ffaebc/users/fcb828b2d60a46389355d1ba91bc44f6/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.903432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbab28-2f5d-4d7c-b3be-c1c1da582c20 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:36.909522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:36.910291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.910648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.911086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] PATH_INFO: `/v3/projects/31343832d13a43609368780f88bc67aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.912393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31343832d13a43609368780f88bc67aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.914337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] RBAC: Authorizing `identity:create_grant(project_id=177434241f6a40a193bf3ff682ffaebc, user_id=fcb828b2d60a46389355d1ba91bc44f6, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:36.915794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.917318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a4c4-2460-4e30-8361-b1f71722d4df admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:36.933151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 938/1898] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:36 2022] PUT /identity/v3/projects/177434241f6a40a193bf3ff682ffaebc/users/fcb828b2d60a46389355d1ba91bc44f6/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:36.944097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:36.945363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:36.945602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:36.978383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] MFA Rules not processed for user `fcb828b2d60a46389355d1ba91bc44f6`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:36.980480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:36.980606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31343832d13a43609368780f88bc67aa', 'request_id': 'req-a2a6073b-aeec-40c0-994d-b07847202244'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.987594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31343832d13a43609368780f88bc67aa', 'request_id': 'req-a2a6073b-aeec-40c0-994d-b07847202244'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:36.995921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31343832d13a43609368780f88bc67aa', 'request_id': 'req-a2a6073b-aeec-40c0-994d-b07847202244'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.011351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] The token cache is being invalidate because project 31343832d13a43609368780f88bc67aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:37.011647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.012294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a2a6073b-aeec-40c0-994d-b07847202244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.013849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 961/1899] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:36 2022] DELETE /identity/v3/projects/31343832d13a43609368780f88bc67aa => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:37.018990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59623a24-ad35-4d31-bf3e-f511ab8a5183 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.019353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59623a24-ad35-4d31-bf3e-f511ab8a5183 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.019700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59623a24-ad35-4d31-bf3e-f511ab8a5183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.037823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:37.045682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-59623a24-ad35-4d31-bf3e-f511ab8a5183 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:37.047384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-59623a24-ad35-4d31-bf3e-f511ab8a5183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.058767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 939/1900] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:37.065775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.067163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.134391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 962/1901] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:37.142182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.142546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.142889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.148390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d38826-839d-469d-80c9-5c79037b1ce1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.153592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.154213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.154496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.154773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.155410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.157948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.171344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:37.173436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.183321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 963/1902] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:37.187460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.189045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 940/1903] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:37.192762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.199702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eac73d-65e7-4d9c-b14d-4775132dfd95 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.227285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:37.264684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.265849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.265849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.266035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.266597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:37.267085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.269550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.270014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73192e-c8d0-4fc0-91f0-3f401655a4fe admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.275644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.276376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.277206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.277589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.278686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.281945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.293102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 964/1904] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:47:37.296360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.316062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 941/1905] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:37.317564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.324081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa41e4-551b-44c9-b02a-05fe86060b87 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.328678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8510f-b500-4697-a65e-3cf7f9dc89a2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.333369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.334081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.334373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.334671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.335449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.337380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.339404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.341011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.341204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.341624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.343264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.345516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.360002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 942/1906] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:37.364273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.373736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 965/1907] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:37.374660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a94e-0af1-48bb-beb6-0aae504beb56 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.378394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.441495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:37.460650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.461157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.461534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.461819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.462468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.464125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.470669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d79b2-bbeb-47d0-a17f-c702cafd92dd admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.475368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 943/1908] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:37.477296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.478167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.478523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.478907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.479552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.479660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.481730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.495196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2343-795b-4a4d-a597-5252637e3efe admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.501070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.501751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.502038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.502317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] PATH_INFO: `/v3/projects/0545e6afde93421b80b9d26b77a543d3/users/73e2ee99f63149b49941e87ca31e22ec/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.510522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 966/1909] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:37.514819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.519631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0545e6afde93421b80b9d26b77a543d3, user_id=73e2ee99f63149b49941e87ca31e22ec, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.522621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.528154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd52e3-a87a-4904-b1a4-35167ede5a94 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.534010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.534541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.535317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.535625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.536326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.538292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.549760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 967/1910] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:37.550771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.553795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.566647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 944/1911] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:37 2022] PUT /identity/v3/projects/0545e6afde93421b80b9d26b77a543d3/users/73e2ee99f63149b49941e87ca31e22ec/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:37.566647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.569071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ca4ac-a551-4583-84f7-fccf7dd8d55d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.641568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb90-d895-492b-9429-91719e4d2d8f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.643526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.644143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.644431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.644711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] PATH_INFO: `/v3/users/fb7852b66e6a4b44a62349887947e727` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.645884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7852b66e6a4b44a62349887947e727)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.647823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.651505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.652218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.652607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.653487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.654298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.656033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d0506-0af3-4f94-ab8c-1dd11d091e24 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.671597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 945/1912] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:37.673226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717498-fdbf-4603-843e-30938431adb6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.673523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717498-fdbf-4603-843e-30938431adb6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.673818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717498-fdbf-4603-843e-30938431adb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.679971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:37.707545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f1717498-fdbf-4603-843e-30938431adb6 None None] MFA Rules not processed for user `73e2ee99f63149b49941e87ca31e22ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:37.710102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f1717498-fdbf-4603-843e-30938431adb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.713983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7852b66e6a4b44a62349887947e727'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.714589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7852b66e6a4b44a62349887947e727'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.728805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7852b66e6a4b44a62349887947e727'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.737929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7852b66e6a4b44a62349887947e727'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.747191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7852b66e6a4b44a62349887947e727'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:37.757664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 968/1913] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:37 2022] DELETE /identity/v3/users/fb7852b66e6a4b44a62349887947e727 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:37.757664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.757664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.822946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 946/1914] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:37.830685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.830685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.830685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.837264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c7df0-5284-4212-bfd2-d63a55624218 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.844675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.845384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.845667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.845943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] PATH_INFO: `/v3/projects/2288aa5924a540ad81f04211e8b7a171/users/6be0ed31ede54d3bb08a59d08e9abb0c/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.852516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:37.853854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.863378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 947/1915] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:37.863965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2288aa5924a540ad81f04211e8b7a171, user_id=6be0ed31ede54d3bb08a59d08e9abb0c, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.864139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.865496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.866689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.880492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc2cf9-371c-42b4-a6c9-35bb90b1919b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.886422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 969/1916] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:37 2022] PUT /identity/v3/projects/2288aa5924a540ad81f04211e8b7a171/users/6be0ed31ede54d3bb08a59d08e9abb0c/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:37.890913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:37.894196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:37.932860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:37.956369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.957182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.957533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.957896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.958698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.960787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.960906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a4e02-c124-40da-aacb-9e395685285a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:37.964935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:37.965659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.965938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.966211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.976398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:37.977661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77b05e-538a-4b76-8fbd-5fdb43a5fdc5 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:37.989570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 970/1917] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:37.990906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:37.991201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:37.991484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:37.994405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 948/1918] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:37.998148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.025609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] MFA Rules not processed for user `6be0ed31ede54d3bb08a59d08e9abb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:38.027159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.040797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:38.066122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e31f38-9bbe-425d-9c84-f304221b5463 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.078074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.078730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.079009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.079281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.080043 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.082088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.115465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 949/1919] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:38.116473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.117850 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 971/1920] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:38.122129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.128703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef103d-d08f-4054-99ee-3f969e9e9a69 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.134078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310a626-fedd-45d2-99a4-4cc108e5a17d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.135066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.135839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.136190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.136532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.137284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.138974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.139680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.140223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.140710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.141017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.141993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.143921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.153612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 950/1921] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:38.158062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.168986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706bca1-bca3-4cec-952c-e09e04103acb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.174740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.175428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.175761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.176046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] PATH_INFO: `/v3/projects/ec9656258d5d43388e50dc14ad5845a5/users/14678f0e78c14f21890b690f367486c7/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.180254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 972/1922] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:38 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:38.185110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.194135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9656258d5d43388e50dc14ad5845a5, user_id=14678f0e78c14f21890b690f367486c7, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.196373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.211530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 951/1923] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:38 2022] PUT /identity/v3/projects/ec9656258d5d43388e50dc14ad5845a5/users/14678f0e78c14f21890b690f367486c7/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:38.215392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.263945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec7eea-c45c-4ae8-b087-76334ea7c294 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.290497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58ce4f-5628-4d30-9dd8-fa7ffbc86d6e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.296457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.297641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.298068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.298183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.298804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.300450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.314754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 952/1924] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:38.315787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.327927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e338-76c2-41a6-b72f-7236ddb8a87d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.333752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.334779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.335136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.335325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] PATH_INFO: `/v3/projects/ec9656258d5d43388e50dc14ad5845a5/users/14678f0e78c14f21890b690f367486c7/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.337473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9656258d5d43388e50dc14ad5845a5, user_id=14678f0e78c14f21890b690f367486c7, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.340284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.343351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.344803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.345285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.345728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.346691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.349802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8434-5edf-4359-8b5d-bec162f29388 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.366349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 953/1925] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:38 2022] PUT /identity/v3/projects/ec9656258d5d43388e50dc14ad5845a5/users/14678f0e78c14f21890b690f367486c7/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:38.366349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.368147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.372272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 973/1926] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:38 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:38.373744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.374099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.374462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.381197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:38.405675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] MFA Rules not processed for user `14678f0e78c14f21890b690f367486c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:38.407416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.420557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:38.439746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3269e-a4a5-46cd-a305-9c66922cc599 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.445380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.446019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.446295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.446567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] PATH_INFO: `/v3/projects/e36a303c592846d397857ef7fc796bc6` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.447725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36a303c592846d397857ef7fc796bc6)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.449733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.482279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 974/1927] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:38.483116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.504691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df0e2-925c-4709-ab2b-4564da47256c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.505748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36a303c592846d397857ef7fc796bc6', 'request_id': 'req-f8d35596-69f5-4dbb-b41a-e6754981c6bb'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.513457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36a303c592846d397857ef7fc796bc6', 'request_id': 'req-f8d35596-69f5-4dbb-b41a-e6754981c6bb'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.521107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36a303c592846d397857ef7fc796bc6', 'request_id': 'req-f8d35596-69f5-4dbb-b41a-e6754981c6bb'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.532621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] The token cache is being invalidate because project e36a303c592846d397857ef7fc796bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:38.532917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.533695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.535858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 954/1928] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:38 2022] DELETE /identity/v3/projects/e36a303c592846d397857ef7fc796bc6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:38.536381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.537688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.563877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.564567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.564764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.565056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.565691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.567303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.581296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 975/1929] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:38.582185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.583823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.596720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2830-1d86-4180-afbb-97bf4ce3bc8b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.601501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.602133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.602411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.602678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] PATH_INFO: `/v3/users/a63e3bf614e2469b9f795e3c9580794e` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.603934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63e3bf614e2469b9f795e3c9580794e)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.605896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.610454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35596-69f5-4dbb-b41a-e6754981c6bb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.615554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.616292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.616660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.617066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.618938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.659192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63e3bf614e2469b9f795e3c9580794e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.659997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63e3bf614e2469b9f795e3c9580794e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.671915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63e3bf614e2469b9f795e3c9580794e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.677952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:38.679030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63e3bf614e2469b9f795e3c9580794e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.687320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63e3bf614e2469b9f795e3c9580794e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:38.694821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 976/1930] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:38 2022] DELETE /identity/v3/users/a63e3bf614e2469b9f795e3c9580794e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:38.696162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.698829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.702132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.709928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.726775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 955/1931] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:38.730548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:38.780902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.782489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.790859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42573-6a18-4bd3-8cb8-a5dcd1726503 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.794209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bee96e-e218-432f-9890-45e6b8dc0c41 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.795898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.796634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.797686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.798032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] PATH_INFO: `/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9/users/fd830c940227495b95fb180de39ef26c/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.826475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca193bd1a6e4f67bddb42c85a1e34d9, user_id=fd830c940227495b95fb180de39ef26c, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.829823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.864329 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 977/1932] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:38 2022] PUT /identity/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9/users/fd830c940227495b95fb180de39ef26c/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:38.869241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:38.877478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.878417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.878803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.879172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.881435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:38.969640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.970276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557f3dc-7dca-40bd-b5b3-5a20a0fb6af7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:38.971171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.975355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:38.976181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:38.976531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:38.976932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:38.977692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:38.979632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f2abc-12bb-438e-81ec-ba96a67e3eff admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:38.988414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 956/1933] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:38.992420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 978/1934] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:38.999344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:39.032403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c3040-bac0-4f1c-b1f2-e2b335fe1d5b service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:39.042832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:39.043750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.044621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.045239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] PATH_INFO: `/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9/users/fd830c940227495b95fb180de39ef26c/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.047968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca193bd1a6e4f67bddb42c85a1e34d9, user_id=fd830c940227495b95fb180de39ef26c, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:39.050789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:39.069143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 957/1935] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:38 2022] PUT /identity/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9/users/fd830c940227495b95fb180de39ef26c/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:39.081495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.081787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.082067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.136273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] MFA Rules not processed for user `fd830c940227495b95fb180de39ef26c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:39.138918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.239905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 979/1936] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:39.252491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:39.254348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.258893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:39.266676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.327160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:39.366626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef292f8f-126f-4f2e-adde-fcd38721e13d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:39.368716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd090-3a88-46e1-b9f2-e821742be652 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:39.371275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:39.372022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.372429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.373129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.374960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.375497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:39.376548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.377098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.377553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] PATH_INFO: `/v3/projects/1df110c21e7d4904b76e3bd30d05971b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.379063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df110c21e7d4904b76e3bd30d05971b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:39.381432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:39.435162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:39.467276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df110c21e7d4904b76e3bd30d05971b', 'request_id': 'req-cbc6f735-4ff7-4a36-968e-1152627a4b20'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:39.470997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:39.473280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef697557-fb9c-4fe4-b508-7532935cce33 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:39.478961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df110c21e7d4904b76e3bd30d05971b', 'request_id': 'req-cbc6f735-4ff7-4a36-968e-1152627a4b20'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:39.484261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 980/1937] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:39.488104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df110c21e7d4904b76e3bd30d05971b', 'request_id': 'req-cbc6f735-4ff7-4a36-968e-1152627a4b20'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:39.502644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] The token cache is being invalidate because project 1df110c21e7d4904b76e3bd30d05971b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:39.503159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:39.504424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cbc6f735-4ff7-4a36-968e-1152627a4b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:39.506328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 958/1938] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:39 2022] DELETE /identity/v3/projects/1df110c21e7d4904b76e3bd30d05971b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:39.511332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.511630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.511911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.547559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:39.548902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.624834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 981/1939] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:39.632654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.633010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.633290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.656936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:39.658383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:39.669361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 959/1940] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:39.673397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:39.684473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadf04b-72ea-4f01-9cc0-efd276563b5f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:39.689567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:39.690271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.690760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.840449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.840449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:39.840449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:39.840449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:39.840449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 982/1941] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:39.841710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:39.841710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581c7ae-49d6-408f-b70e-559a58eca2ce None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:39.841710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:39.841710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:39.842064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:39.842064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:39.842064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:39.842064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:39.987166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 960/1942] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:39 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:39.991231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.003632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a509-54f0-47e6-92d5-703c1837a75f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.101709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.102339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.102620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.102893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.103529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.105905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.121872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:40.142922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 983/1943] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:39 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:40.143667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.158339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfb1ff-36a2-4ca0-8a97-56069db3eca4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.166847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.167794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.168299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.168643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.169408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.172784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.188631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 961/1944] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:40.192622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.204563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e7024-717a-4029-9eea-e360fbf18496 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.212143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.213160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.213160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.213388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.214092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.215843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4cfa-d462-469b-85af-94f120ade82b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.230905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 984/1945] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:40.235494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.245810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f88c7-4d3e-4144-9488-3983a201d4c1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.250707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.251378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.251657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.251930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] PATH_INFO: `/v3/projects/9cb3475d04784a778b54aba5166c6105/users/2ac38be871134af9bad54ab67770a1f6/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.269381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb3475d04784a778b54aba5166c6105, user_id=2ac38be871134af9bad54ab67770a1f6, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.271956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.289029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 962/1946] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:40 2022] PUT /identity/v3/projects/9cb3475d04784a778b54aba5166c6105/users/2ac38be871134af9bad54ab67770a1f6/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:40.299489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.299778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.300057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.331822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] MFA Rules not processed for user `2ac38be871134af9bad54ab67770a1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:40.333183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:40.398908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 985/1947] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:40.410504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.412197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:40.471065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19409fbe-dc1c-4e78-8242-d9d9a59f91ba admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.475056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.475672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.475957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.476788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.485493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.486911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a7927-aa6b-458f-9214-44a1c14f653c service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.498984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 963/1948] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:40.520402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.586640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f25f-9265-4ec0-8bc0-fdc57f23b7f4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.591461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.592103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.592411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.592739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] PATH_INFO: `/v3/users/2ac38be871134af9bad54ab67770a1f6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.594376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac38be871134af9bad54ab67770a1f6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.596344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.649064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac38be871134af9bad54ab67770a1f6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:40.649689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac38be871134af9bad54ab67770a1f6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:40.663067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac38be871134af9bad54ab67770a1f6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:40.670326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac38be871134af9bad54ab67770a1f6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:40.673021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:40.679310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac38be871134af9bad54ab67770a1f6'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:40.686953 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 986/1949] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:40 2022] DELETE /identity/v3/users/2ac38be871134af9bad54ab67770a1f6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:40.692120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.692436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.692726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.738139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:40.739104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:40.810521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:40.827412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 964/1950] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:40.832683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:40.834287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:40.904035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae9d3-f038-48a8-9507-993bae19588b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:40.908333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:40.909218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:40.909599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:40.910249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:40.921927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:40.923743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc115e45-9d30-4b40-9c41-0c4419b3de9f service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:40.934992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 987/1951] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:41.273101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.277963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:41.302697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba46a4-626b-41b9-aaaa-073a16c9d3c4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.310360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.311147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.311526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.311892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] PATH_INFO: `/v3/projects/9cb3475d04784a778b54aba5166c6105` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.313672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb3475d04784a778b54aba5166c6105)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.315774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.381617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb3475d04784a778b54aba5166c6105', 'request_id': 'req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:41.388920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb3475d04784a778b54aba5166c6105', 'request_id': 'req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:41.398377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb3475d04784a778b54aba5166c6105', 'request_id': 'req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:41.413389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] The token cache is being invalidate because project 9cb3475d04784a778b54aba5166c6105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:41.413609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:41.414402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0bc29a5d-9ed3-4bc1-83e8-09d55c71acfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:41.416033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 965/1952] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:41 2022] DELETE /identity/v3/projects/9cb3475d04784a778b54aba5166c6105 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:41.421158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.421458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.421809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.464365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:41.465739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:41.600093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 988/1953] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:41.609018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.609426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.609714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.633766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:41.635204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:41.650247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.651058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 966/1954] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:41.662279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861da049-7384-41d3-b1ed-1867ecac5d6d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.668644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.669654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.670232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.670835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.671655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:41.672137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.674165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.695322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.707633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835041c9-abe0-42ee-94f5-5d74caa9ad77 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.708619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 989/1955] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:41.713343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.713949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.714228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.714516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.715844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.719414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.758952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 967/1956] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:41 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:41.766547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.785243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20e7be-5cc6-4df4-9bba-3768fbda08ed admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.890836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.891538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.891818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.892091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.892724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.894767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.916995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 990/1957] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:41 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:41.921099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.931194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da04222-56e8-4ee0-9656-a17a49dc96b6 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.935713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.936311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.936588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.936858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.937524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.939122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.950075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 968/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:41.954487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.964186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1413-f4cc-45e7-8b4e-f69191ecfc07 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:41.969270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:41.969863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:41.970144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:41.970416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:41.971045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:41.972665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603d439-668e-41e6-ad77-b437fb425293 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:41.984721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 991/1959] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:41.989094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:41.998988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813200a1-b493-43cf-8203-2af059da13d5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.003654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.004308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.004594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.004876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] PATH_INFO: `/v3/projects/ddb141ed4e1d4c5d8e41bd42712ed6e0/users/58567ef3bb864a72a7bc7014d9ca25c4/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.023255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb141ed4e1d4c5d8e41bd42712ed6e0, user_id=58567ef3bb864a72a7bc7014d9ca25c4, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:42.026041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:42.058111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 969/1960] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:41 2022] PUT /identity/v3/projects/ddb141ed4e1d4c5d8e41bd42712ed6e0/users/58567ef3bb864a72a7bc7014d9ca25c4/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:42.061532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.061980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.062464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.081941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:42.153150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] MFA Rules not processed for user `58567ef3bb864a72a7bc7014d9ca25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:42.154401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.223224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:42.225092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.293920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 992/1961] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:42.303054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:42.304564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.322128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eb291-5317-4599-bf62-193b44f5a728 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.336981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.336981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.336981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.336981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] PATH_INFO: `/v3/users/fcb828b2d60a46389355d1ba91bc44f6` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.337439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb828b2d60a46389355d1ba91bc44f6)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:42.337439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:42.356994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:42.369074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542242ec-e8dc-4426-b77d-8ee8a59fd23d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.373136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.373761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.374037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.374307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.383582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:42.385072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb828b2d60a46389355d1ba91bc44f6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.385456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269449c0-41dc-43b7-be85-90879ce8026f service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:42.393272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb828b2d60a46389355d1ba91bc44f6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.397452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 993/1962] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:42.409208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb828b2d60a46389355d1ba91bc44f6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.418180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb828b2d60a46389355d1ba91bc44f6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.419268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb828b2d60a46389355d1ba91bc44f6'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.425980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 970/1963] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:42 2022] DELETE /identity/v3/users/fcb828b2d60a46389355d1ba91bc44f6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:42.436082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.436372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.436640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.457174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:42.461181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.467221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:42.469138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.544543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 994/1964] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:42.544728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e264-f599-4946-8757-498bcbae525f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.547708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.548379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.548663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.548941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] PATH_INFO: `/v3/users/6be0ed31ede54d3bb08a59d08e9abb0c` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.550156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be0ed31ede54d3bb08a59d08e9abb0c)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:42.552154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:42.552295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:42.553869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.598868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:42.623291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be0ed31ede54d3bb08a59d08e9abb0c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.624567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecb5f6-e71f-420f-8777-798096c72454 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.628882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.630063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.630225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.630561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.632866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be0ed31ede54d3bb08a59d08e9abb0c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.641782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:42.643334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1301aa2-060c-48bb-b64d-f4417cb271db service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:42.647221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be0ed31ede54d3bb08a59d08e9abb0c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.654722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be0ed31ede54d3bb08a59d08e9abb0c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.655232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 995/1965] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:42.655541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be0ed31ede54d3bb08a59d08e9abb0c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:42.663398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 971/1966] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:42 2022] DELETE /identity/v3/users/6be0ed31ede54d3bb08a59d08e9abb0c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:42.667812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.668621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.671180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:42.704058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:42.705358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.807585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 996/1967] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:42.830754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:42.832674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:42.868677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:42.974908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4c2a0-b364-4461-a1fb-05aa061cc269 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:42.986973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:42.988939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:42.990126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:42.991793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.016855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.018511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.039774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.041462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.075328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 972/1968] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:43.075763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16612e5-bddc-4445-b97d-8fe2aa285f11 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.094915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.095788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.097383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.097911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] PATH_INFO: `/v3/projects/177434241f6a40a193bf3ff682ffaebc` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.101302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=177434241f6a40a193bf3ff682ffaebc)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.103903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.175390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177434241f6a40a193bf3ff682ffaebc', 'request_id': 'req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.186221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177434241f6a40a193bf3ff682ffaebc', 'request_id': 'req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.193990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177434241f6a40a193bf3ff682ffaebc', 'request_id': 'req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.210105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] The token cache is being invalidate because project 177434241f6a40a193bf3ff682ffaebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:43.210389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.211178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.216730 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 997/1969] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/projects/177434241f6a40a193bf3ff682ffaebc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:43.217430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.228701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11176f3b-7a86-4a26-b000-317ea5f5cc79 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.291349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.292020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.292306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.292584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] PATH_INFO: `/v3/users/73e2ee99f63149b49941e87ca31e22ec` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.293769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e2ee99f63149b49941e87ca31e22ec)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.295689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.350094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e2ee99f63149b49941e87ca31e22ec'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.360116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e2ee99f63149b49941e87ca31e22ec'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.378657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e2ee99f63149b49941e87ca31e22ec'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.387803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e2ee99f63149b49941e87ca31e22ec'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.388582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e2ee99f63149b49941e87ca31e22ec'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.395774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 973/1970] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/users/73e2ee99f63149b49941e87ca31e22ec => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:43.494352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.495811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.516173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c70e02-5e1a-42c2-a1b4-a987d1142eb8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.608833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.610050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.610331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.610604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] PATH_INFO: `/v3/projects/2288aa5924a540ad81f04211e8b7a171` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.611862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2288aa5924a540ad81f04211e8b7a171)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.614071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.617349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:43.685430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2288aa5924a540ad81f04211e8b7a171', 'request_id': 'req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.692245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2288aa5924a540ad81f04211e8b7a171', 'request_id': 'req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.701435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2288aa5924a540ad81f04211e8b7a171', 'request_id': 'req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.721554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] The token cache is being invalidate because project 2288aa5924a540ad81f04211e8b7a171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:43.721801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.725405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.729126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 998/1971] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/projects/2288aa5924a540ad81f04211e8b7a171 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:43.731312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.733000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.747817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba271f9-485f-4f86-a389-c6709b66886d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.793614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:43.829151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.832397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.832852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.833352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] PATH_INFO: `/v3/users/fd830c940227495b95fb180de39ef26c` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.834811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd830c940227495b95fb180de39ef26c)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.837399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.868694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.870702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.892368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0de6a-c2b1-48bd-a1a2-969b4f1e00b8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.898078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.898727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.898999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.899273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] PATH_INFO: `/v3/projects/0545e6afde93421b80b9d26b77a543d3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.900398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0545e6afde93421b80b9d26b77a543d3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.902379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.908815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd830c940227495b95fb180de39ef26c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.920699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd830c940227495b95fb180de39ef26c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.937030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd830c940227495b95fb180de39ef26c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.946169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd830c940227495b95fb180de39ef26c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.946921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd830c940227495b95fb180de39ef26c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.953646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 974/1972] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/users/fd830c940227495b95fb180de39ef26c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:43.959998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0545e6afde93421b80b9d26b77a543d3', 'request_id': 'req-27eb5e2d-0e48-490f-8634-deb0c128ba0e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.968244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0545e6afde93421b80b9d26b77a543d3', 'request_id': 'req-27eb5e2d-0e48-490f-8634-deb0c128ba0e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.974813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0545e6afde93421b80b9d26b77a543d3', 'request_id': 'req-27eb5e2d-0e48-490f-8634-deb0c128ba0e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.988042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] The token cache is being invalidate because project 0545e6afde93421b80b9d26b77a543d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:43.988280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.989489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:43.991059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 999/1973] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/projects/0545e6afde93421b80b9d26b77a543d3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:43.995081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.996799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.080115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d3d0f-cbc2-40b6-82e6-286d33fa50c4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.091920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.097595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.097926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.098622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] PATH_INFO: `/v3/users/14678f0e78c14f21890b690f367486c7` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.101246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=14678f0e78c14f21890b690f367486c7)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.103565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.180414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14678f0e78c14f21890b690f367486c7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.194843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14678f0e78c14f21890b690f367486c7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.201096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.217347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14678f0e78c14f21890b690f367486c7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.237220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14678f0e78c14f21890b690f367486c7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.238210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6033198e-e943-4e01-8caa-2588605ab66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14678f0e78c14f21890b690f367486c7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.257239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 975/1974] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:43 2022] DELETE /identity/v3/users/14678f0e78c14f21890b690f367486c7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:44.336357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.339120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.429337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb5e2d-0e48-490f-8634-deb0c128ba0e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.450711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.498658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.499224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.499602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.500010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] PATH_INFO: `/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.501173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca193bd1a6e4f67bddb42c85a1e34d9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.503231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.568983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca193bd1a6e4f67bddb42c85a1e34d9', 'request_id': 'req-68770c69-6e62-4604-abfa-875a71ac1f81'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.576204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca193bd1a6e4f67bddb42c85a1e34d9', 'request_id': 'req-68770c69-6e62-4604-abfa-875a71ac1f81'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.583946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca193bd1a6e4f67bddb42c85a1e34d9', 'request_id': 'req-68770c69-6e62-4604-abfa-875a71ac1f81'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.600321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] The token cache is being invalidate because project cca193bd1a6e4f67bddb42c85a1e34d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:44.600613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.601352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-68770c69-6e62-4604-abfa-875a71ac1f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.602856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1000/1975] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:44 2022] DELETE /identity/v3/projects/cca193bd1a6e4f67bddb42c85a1e34d9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:44.607891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.610340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.610481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.640984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.642643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.711101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 976/1976] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.719256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.719772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.719849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.741111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.742579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.752671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1001/1977] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.758282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.771362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef361910-7ea0-4722-b53e-946430ebd743 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.776638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.777300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.777587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.777870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.778528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.778836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.779155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.780583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.780707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.805195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939a975-efbb-4e7b-ab94-f488202d7afe None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.805195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.805195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.805195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.805755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] PATH_INFO: `/v3/projects/ec9656258d5d43388e50dc14ad5845a5` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.806992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9656258d5d43388e50dc14ad5845a5)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.810206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 977/1978] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.810795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.817304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.829617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494f9a4-b1ad-4dd0-aefa-d27b3c1ed6f1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.834521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.835125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.835532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.835815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.836468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.838430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.870412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9656258d5d43388e50dc14ad5845a5', 'request_id': 'req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.873134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 978/1979] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.874569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.879070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9656258d5d43388e50dc14ad5845a5', 'request_id': 'req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.886450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9656258d5d43388e50dc14ad5845a5', 'request_id': 'req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.906445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] The token cache is being invalidate because project ec9656258d5d43388e50dc14ad5845a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:44.906959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.907811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2cc9f8f6-6f53-44ff-bdce-963fc7c37414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:44.910026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1002/1980] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:44 2022] DELETE /identity/v3/projects/ec9656258d5d43388e50dc14ad5845a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:44.916722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5d58-4015-4865-9df3-989033c42891 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.917188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5d58-4015-4865-9df3-989033c42891 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.917473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5d58-4015-4865-9df3-989033c42891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.940898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1dfa5d58-4015-4865-9df3-989033c42891 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.942164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1dfa5d58-4015-4865-9df3-989033c42891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.958369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162bd1e-cf35-4d31-99d1-385c6251b731 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.964114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.964780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.965141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.965575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.966220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.968266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.972926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.994491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 979/1981] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:44.998470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.000999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.018946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6374a-acdb-4b2d-b87d-f8577bfe20fa admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.021306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1003/1982] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.028245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.028880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.029211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.029489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.029963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.030252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.030370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.030531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.031947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.049018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 980/1983] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.052870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:45.056351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.062808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.065651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1004/1984] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.071443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.079567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d997f-7a5c-48c5-8fa3-1d72d6908505 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.080558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad66d0-2217-44be-b8d7-a455a00051dd None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.084561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.085232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.085457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.085581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.085787 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.086055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.086340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.086452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.086614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.087258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:45.087562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.087999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.089194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.099287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 981/1985] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.103111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1005/1986] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:47:45.105988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.106275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.116863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f87ee3-43db-446f-94fd-d9a29562b6df admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.117088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa5899-491b-4a32-b99b-582ca54747ce admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.121470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.121618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.122099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.122208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.122425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.122530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.122699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] PATH_INFO: `/v3/projects/84ac9e8e366e4839919c7bebf8c369d6/users/9895e872819b475094a1921967256e40/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.122803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.123388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.125969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.131310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.140677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ac9e8e366e4839919c7bebf8c369d6, user_id=9895e872819b475094a1921967256e40, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.143425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deb74b-8d87-46e1-b192-1605f79ec59f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.163775 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 982/1987] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.165340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.173751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1006/1988] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/84ac9e8e366e4839919c7bebf8c369d6/users/9895e872819b475094a1921967256e40/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:45.175462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.175742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.176007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.214448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] MFA Rules not processed for user `9895e872819b475094a1921967256e40`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:45.215903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.242732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f92617-fda1-4a3c-bc5b-1cf7ee9e79a0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.249429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.250077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.250369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.250646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.251286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.253248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.274613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 983/1989] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:45.278661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.289054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebe7da-719b-4211-8a52-f698029f4d55 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.290057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1007/1990] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.294677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.295394 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.295566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.295844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.296563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.298761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.301247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.302624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.313418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 984/1991] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:45.314916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.325455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e1ae3-b98a-440b-9484-d27db511e736 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.332272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.332885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.333220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.333496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.334136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.335863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.347622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 985/1992] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.352031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.371509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf088040-1ab0-4f60-8524-d930c30e4c5a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.372253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a63922-d1ca-4714-ae95-5ec29fedd0bf None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.377055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.377934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.378417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.378574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.381113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.381113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.381113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.381113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] PATH_INFO: `/v3/projects/3292df9f2557486ba42263f2db9f0d80/users/34cd297bb44243f2ba73eaa59c016c23/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.399260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.400791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2df912-4f13-4c65-9044-0ce2cf30dbf3 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.411850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3292df9f2557486ba42263f2db9f0d80, user_id=34cd297bb44243f2ba73eaa59c016c23, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.413137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1008/1993] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:45.417033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.435272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 986/1994] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/3292df9f2557486ba42263f2db9f0d80/users/34cd297bb44243f2ba73eaa59c016c23/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:45.445367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.445771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.446009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.491865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] MFA Rules not processed for user `34cd297bb44243f2ba73eaa59c016c23`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:45.493125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.499174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.584320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1009/1995] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.589345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.669249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6839d-723c-4028-993b-55c2c2328a10 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.674836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.675600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.675906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.676183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.676898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.678831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.725233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 987/1996] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.726434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.738577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85d231-d7d6-4cca-b1c9-2f9941d90928 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.805364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.805983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.806342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.806615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.807345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.809396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.838039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1010/1997] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.842270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.853579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa6e67-2115-4cd6-a9d8-4810e0c14f1f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.858396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.859112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.859463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.859805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.860557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.862265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.875997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 988/1998] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.880175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.890879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ce251-0959-4ecf-8b2d-cfd4095425fc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.895468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.896121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.896400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.896673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] PATH_INFO: `/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8/users/fbcd41e874b647d98e63b1cf8a54382e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.912358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47cdae2311e42a6b5dfa3c5d77375a8, user_id=fbcd41e874b647d98e63b1cf8a54382e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.915050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.928441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1011/1999] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8/users/fbcd41e874b647d98e63b1cf8a54382e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:45.932640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.939715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.944154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b484e34-1c7d-49fd-8667-090dc6d4d172 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.002239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.002854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.003137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.003407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.004069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.005682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca65db-c54b-4136-8189-349504753bda admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.015289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 989/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:46.020131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.030173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708e9a6-f428-4643-a20c-3bece0aa48c6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.034796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.035435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.035713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.035984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] PATH_INFO: `/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8/users/fbcd41e874b647d98e63b1cf8a54382e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.037880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47cdae2311e42a6b5dfa3c5d77375a8, user_id=fbcd41e874b647d98e63b1cf8a54382e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.040487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.050678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1012/2001] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:46 2022] PUT /identity/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8/users/fbcd41e874b647d98e63b1cf8a54382e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:46.060634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.060863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.061172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.097794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] MFA Rules not processed for user `fbcd41e874b647d98e63b1cf8a54382e`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.099918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.179954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 990/2002] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.188271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.286727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a24036-db48-40cb-b254-eb255e53717d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.290535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.291153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.291438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.291750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.304257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:46.365234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.366729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f828ca6-f65c-405e-be0f-fb470f68be77 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.385655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1013/2003] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:49.751057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:49.816469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e686a6-061b-4ce7-b17c-8401eabca875 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.821353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.822042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.822322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.822596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] PATH_INFO: `/v3/users/9895e872819b475094a1921967256e40` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.823766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9895e872819b475094a1921967256e40)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.825719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.856768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:49.882089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9895e872819b475094a1921967256e40'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:49.890357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9895e872819b475094a1921967256e40'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:49.903014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9895e872819b475094a1921967256e40'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:49.915298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9895e872819b475094a1921967256e40'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:49.916208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9895e872819b475094a1921967256e40'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:49.923859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 991/2004] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:49 2022] DELETE /identity/v3/users/9895e872819b475094a1921967256e40 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:49.928355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.928898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.928898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.948715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:49.950225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.043085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1014/2005] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:50.081473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:50.082638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.157581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b654c-e705-4f31-a9a3-dee6775e54a0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:50.161611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:50.162257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.162538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.162813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.172053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:50.173574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233d1b-97e9-44a7-baa1-ff9f9e05e527 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:50.183988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 992/2006] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:50.652660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:50.656320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.690623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867ce1b-1a84-4336-9604-2f29a0ed5814 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:50.706591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:50.707773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.710522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.711009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] PATH_INFO: `/v3/projects/84ac9e8e366e4839919c7bebf8c369d6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.715096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ac9e8e366e4839919c7bebf8c369d6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:50.717746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:50.837446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ac9e8e366e4839919c7bebf8c369d6', 'request_id': 'req-909ed58a-6807-4b74-aad0-223b3a832a2c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.853570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ac9e8e366e4839919c7bebf8c369d6', 'request_id': 'req-909ed58a-6807-4b74-aad0-223b3a832a2c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.868298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ac9e8e366e4839919c7bebf8c369d6', 'request_id': 'req-909ed58a-6807-4b74-aad0-223b3a832a2c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.901512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] The token cache is being invalidate because project 84ac9e8e366e4839919c7bebf8c369d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:50.901808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.902557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-909ed58a-6807-4b74-aad0-223b3a832a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.915683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1015/2007] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:50 2022] DELETE /identity/v3/projects/84ac9e8e366e4839919c7bebf8c369d6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:50.917479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.917984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.918190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.957487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:50.959832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:51.063058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 993/2008] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:51.071512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.071995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.072072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.101986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:51.103304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:51.114258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1016/2009] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:51.118929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.129244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f61994-036c-4241-a8c0-d32c12e196b5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.135540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.136210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.136496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.136727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.137398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:51.137704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.139884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.153051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 994/2010] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:51.157865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.167126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b9f0a-d89a-459e-a9c6-39896239896d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.171681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.172281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.172556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.172825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.173506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.175393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.217607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1017/2011] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:51.221543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.234174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18dd1-c21f-49f0-9916-cf3419b6c765 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.307670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:51.346624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.347484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.347778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.348058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.348715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.350719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.377973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 995/2012] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:51.379479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.390649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024ea1-8935-45ff-93b3-b3871599879c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.395185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.395782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.396059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.396331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.396970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.398589 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.400784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:51.410341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1018/2013] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:51.414094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.424768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec51a2b-782e-405d-a845-ed9add71f7eb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.429427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.430035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.430315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.430590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.431226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.432789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94537956-2b1d-46a3-8dd8-063ec06eb448 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.443763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 996/2014] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:51.448040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.459322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177a70d-720c-49a9-a942-a786b7c195c2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.464861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.465646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.465929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.466208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] PATH_INFO: `/v3/projects/052c415fcbd34eebbd5b50af4e10a74f/users/e7eef48bc4fd41008beffc3dc32de757/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.482592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=052c415fcbd34eebbd5b50af4e10a74f, user_id=e7eef48bc4fd41008beffc3dc32de757, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.485395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.500013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1019/2015] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:51 2022] PUT /identity/v3/projects/052c415fcbd34eebbd5b50af4e10a74f/users/e7eef48bc4fd41008beffc3dc32de757/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:51.510587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.510887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.511168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.542092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] MFA Rules not processed for user `e7eef48bc4fd41008beffc3dc32de757`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:51.543387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:51.622844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 997/2016] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:51.627842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.716028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d6a2-3f1a-489f-9866-115d1024b98f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.722892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.723330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.723614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.723885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.724673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.726647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.769139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1020/2017] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:51.769896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.786790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0820af5-2ca4-44c7-a3bc-d3c5d6748e0b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.852564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.853229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.853510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.853815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.854516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.856419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.879636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 998/2018] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:51 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:51.883873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.894766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca148332-81b5-4d20-b6d1-28897fbf4525 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.910470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.910470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.910470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.910470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.910924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.910924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.925421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1021/2019] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:51.926361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.936533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a2c1c-29d3-490c-afb2-388b9207c373 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.941432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.942014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.942363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.942703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:51.943360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:51.945403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a325-ddf1-42a4-a331-043a0b53b269 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:51.964527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 999/2020] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:51.966279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.980010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98913b-2eda-412c-84f4-c0b03a199c73 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:51.986721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:51.987659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:51.988406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:51.988834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] PATH_INFO: `/v3/projects/97e7a229a45149e8a66dc60fb70b5af3/users/15ff53cba0674db4b8d42516e3fc2c69/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.011553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e7a229a45149e8a66dc60fb70b5af3, user_id=15ff53cba0674db4b8d42516e3fc2c69, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.014287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.044069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1022/2021] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:51 2022] PUT /identity/v3/projects/97e7a229a45149e8a66dc60fb70b5af3/users/15ff53cba0674db4b8d42516e3fc2c69/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:52.057137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.057558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.057840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.111725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.113565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.120550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] MFA Rules not processed for user `15ff53cba0674db4b8d42516e3fc2c69`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:52.123176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.141381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:52.221267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b99bef-4c14-4d03-bac0-8e6da73900b9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.221770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1000/2022] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:52.225783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.231139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.232074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.232074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.232307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.234763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.282480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:52.326753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50d07-c76c-4d78-a606-43287a428d96 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.329571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.331630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.333237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.334411 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.334722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.334982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.335896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.338900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.349383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1023/2023] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:52.377358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1001/2024] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:52.378952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.395086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfd003-1773-4e59-8119-530d4dd59aa2 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.473028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.473606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.473863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.474570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.477584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.479754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.522190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1024/2025] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:52.523486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.537327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ff27c-029d-4d7d-8c21-3c9706b4968c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.542445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.543186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.543490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.543675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.545705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.546418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.563131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1002/2026] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:52.564014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.576374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628068a6-401b-40b1-b30f-25ea9fadb04a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.582446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.583359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.583529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.583761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] PATH_INFO: `/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa/users/cb09dc07110740bc8ec7e9f7db2d8e7f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.601675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d43400742db4aaa8c3a9fb1c3f548aa, user_id=cb09dc07110740bc8ec7e9f7db2d8e7f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.604857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.620976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1025/2027] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:52 2022] PUT /identity/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa/users/cb09dc07110740bc8ec7e9f7db2d8e7f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:52.625199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.637572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfbd22-ab22-4258-b6d6-4b9063f50606 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.765895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.766532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.766994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.767327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.768154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.776091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b177d-02d7-44af-b0bf-b4af148e85a9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.793364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1003/2028] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:52.797945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.830791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4a71-47e3-4796-8a94-dc4c1c373f70 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.836215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.837607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.838086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.838460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] PATH_INFO: `/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa/users/cb09dc07110740bc8ec7e9f7db2d8e7f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.840806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d43400742db4aaa8c3a9fb1c3f548aa, user_id=cb09dc07110740bc8ec7e9f7db2d8e7f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.844659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.855599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1026/2029] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:52 2022] PUT /identity/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa/users/cb09dc07110740bc8ec7e9f7db2d8e7f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:52.867379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.867671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.867954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.896892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] MFA Rules not processed for user `cb09dc07110740bc8ec7e9f7db2d8e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:52.898158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.908275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:52.971226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1004/2030] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:52.982786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.985478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.045339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c8b12-0cf8-44f1-81cc-d42b38caa3f9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.049497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.050103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.050379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.050650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.090363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:53.106926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.108372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.117621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1027/2031] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:55.735662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:55.747277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dbd5d-1677-4d26-99ee-fb46772c205e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:55.767780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:55.768533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:55.768816 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:55.769125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:55.841331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:55.843746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:55.859186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1005/2032] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:56.603965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.614239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef70a2-7626-449d-a193-37c333a891e3 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.622710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.623596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.623878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.624189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.636265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.637908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3909326-0e4e-4198-8501-7f822e9eed2b service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.651493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1028/2033] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:58.485470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.486999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.555450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774633-2293-43fc-aa6c-084fd38d1647 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.559973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.560608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.560886 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.561188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] PATH_INFO: `/v3/users/34cd297bb44243f2ba73eaa59c016c23` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.562841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34cd297bb44243f2ba73eaa59c016c23)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.564750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.626203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34cd297bb44243f2ba73eaa59c016c23'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:58.627976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:58.634613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34cd297bb44243f2ba73eaa59c016c23'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:58.647407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34cd297bb44243f2ba73eaa59c016c23'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:58.654611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34cd297bb44243f2ba73eaa59c016c23'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:58.655341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34cd297bb44243f2ba73eaa59c016c23'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:58.662288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1006/2034] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:58 2022] DELETE /identity/v3/users/34cd297bb44243f2ba73eaa59c016c23 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:58.666572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.667019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.667108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.687067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:58.688284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.751302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1029/2035] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:58.760992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.762431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.822090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71214b91-6b53-4d28-9135-158eab0d5729 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.825827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.826502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.826840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.827147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.837132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.838576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.847983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1007/2036] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:59.244677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.246138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:59.258471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2df10-9023-4f04-b2a1-084109f012fa None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.263211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.263972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.264255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.264562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] PATH_INFO: `/v3/projects/3292df9f2557486ba42263f2db9f0d80` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.266056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3292df9f2557486ba42263f2db9f0d80)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.268834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.321405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3292df9f2557486ba42263f2db9f0d80', 'request_id': 'req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.328390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3292df9f2557486ba42263f2db9f0d80', 'request_id': 'req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.337360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3292df9f2557486ba42263f2db9f0d80', 'request_id': 'req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.356831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] The token cache is being invalidate because project 3292df9f2557486ba42263f2db9f0d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:59.357641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.360096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.362726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1030/2037] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:59 2022] DELETE /identity/v3/projects/3292df9f2557486ba42263f2db9f0d80 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:59.367648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.382230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d110ae9-d699-4782-8e29-c314fbd48d4e service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.466602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.468419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.468804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.469181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] PATH_INFO: `/v3/users/fbcd41e874b647d98e63b1cf8a54382e` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.470528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcd41e874b647d98e63b1cf8a54382e)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.473198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.553809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcd41e874b647d98e63b1cf8a54382e'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.567414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcd41e874b647d98e63b1cf8a54382e'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.587433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:59.592172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcd41e874b647d98e63b1cf8a54382e'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.612997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcd41e874b647d98e63b1cf8a54382e'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.614556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8fa5c1bb-5684-4c34-a796-11b515764439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcd41e874b647d98e63b1cf8a54382e'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:59.622515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1008/2038] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:59 2022] DELETE /identity/v3/users/fbcd41e874b647d98e63b1cf8a54382e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:00.112895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.114421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:00.127724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f4ea-26d6-4b4b-861c-ee44773dfa72 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.206283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:00.245058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.245727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.246013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.246290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] PATH_INFO: `/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.263178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47cdae2311e42a6b5dfa3c5d77375a8)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.265325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.373609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47cdae2311e42a6b5dfa3c5d77375a8', 'request_id': 'req-7e1654b3-f639-4a49-9124-9a53a296b524'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:00.405810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47cdae2311e42a6b5dfa3c5d77375a8', 'request_id': 'req-7e1654b3-f639-4a49-9124-9a53a296b524'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:00.426681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47cdae2311e42a6b5dfa3c5d77375a8', 'request_id': 'req-7e1654b3-f639-4a49-9124-9a53a296b524'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:00.452321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] The token cache is being invalidate because project c47cdae2311e42a6b5dfa3c5d77375a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:00.452624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:00.453358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7e1654b3-f639-4a49-9124-9a53a296b524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:00.459923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1031/2039] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:00 2022] DELETE /identity/v3/projects/c47cdae2311e42a6b5dfa3c5d77375a8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:00.461871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.462163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.462444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.494446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:00.496036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:00.621319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1009/2040] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:00.631945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.632234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.632509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.653858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:00.655350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:00.665004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1032/2041] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:00.667648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.679757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805532f-f3c4-4424-b01c-c4e1d877c09a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.684582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.685241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.685519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.685793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.687235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:00.687539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.689483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.707845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1010/2042] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:00.712079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.726382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2217c4-4efa-418f-a1a3-e14748ad602e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.738371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.739021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.748391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.749794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.753244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.761505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.819286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1033/2043] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:00.823342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.834497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d907f-7c98-447a-bbdb-f6ea384c9d4f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.994783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.995425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.995710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.995988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.996633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.998615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.023336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1011/2044] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:01.027468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.038916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229dc6-770f-4493-8e2d-bd46b99f53e7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.045011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.045011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.045011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.045011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.046206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.047963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.059607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1034/2045] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:01.064477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.082795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e35b83-1185-4f01-8259-81b326503e76 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.088412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.089061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.089349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.089625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.090393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.092049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53367078-15e3-47d5-8596-6630b88d37ce admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.107199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1012/2046] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:01.111817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.123976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e8a9-91dc-42df-9820-e72e053d256d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.129412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.130229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.130520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.130822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] PATH_INFO: `/v3/projects/e4f2d720a5ef47c995eaec110c676761/users/33ef9645a7834dfda95b1d616854390a/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.151287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f2d720a5ef47c995eaec110c676761, user_id=33ef9645a7834dfda95b1d616854390a, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.153977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.172126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1035/2047] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:01 2022] PUT /identity/v3/projects/e4f2d720a5ef47c995eaec110c676761/users/33ef9645a7834dfda95b1d616854390a/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:01.183856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.184391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.184889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.221231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] MFA Rules not processed for user `33ef9645a7834dfda95b1d616854390a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:01.222713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.294601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.332041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1013/2048] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:01.339757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.418887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf480-cfcd-4d40-8791-38812ec634bd admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.425465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.426028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.426376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.426856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.427566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.429734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.469186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1036/2049] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:01.470890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.483534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cc424-d938-4c0f-9ceb-6d69d26bab54 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.553361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.553796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.554083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.554361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.554997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.556919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.581261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1014/2050] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:01.584986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.591090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.598418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a119a9f-00f9-4800-a8e3-8df3bb51af6b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.605989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.606804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.608048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.609042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.609773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.611503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.622715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1037/2051] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:01.626486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.637142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a5f30-1fa0-4aa2-98d0-5dafb31f8384 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.643052 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.643761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.644048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.644323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.645003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.646591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4649-127e-42c0-b7c6-8274086f8ab1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.658433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1015/2052] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:01.662517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.672761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f4b5-7e2a-4bd0-8fb2-42c832ba809f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.677956 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.678251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.678533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.678812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] PATH_INFO: `/v3/projects/840f194741544f298a5bbe3f2b1c7f13/users/a992cd1260274c69a2cf5691139420c1/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.695039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=840f194741544f298a5bbe3f2b1c7f13, user_id=a992cd1260274c69a2cf5691139420c1, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.698749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.717083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1038/2053] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:01 2022] PUT /identity/v3/projects/840f194741544f298a5bbe3f2b1c7f13/users/a992cd1260274c69a2cf5691139420c1/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:01.729833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.730219 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.730409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.762459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] MFA Rules not processed for user `a992cd1260274c69a2cf5691139420c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:01.763685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.843864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1016/2054] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:01.848541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.890380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.891621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.921906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79530-875c-4c3f-a253-7b966055f27f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.926822 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.927441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.927810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.928089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.928724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.930773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.958817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.962600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1039/2055] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:01.966375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.975342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817a943-3be0-4cdd-a9e6-7e9f4b377b09 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.039929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c22e4-21bf-42d3-96b7-f632770eed11 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.046782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.047402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.047673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.047946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.048588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.048975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.049636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.049921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.050203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] PATH_INFO: `/v3/users/e7eef48bc4fd41008beffc3dc32de757` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.050571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.051421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7eef48bc4fd41008beffc3dc32de757)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.053655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.078404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1040/2056] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:02.082316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.097114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7bdc0-77ec-427d-a27a-19133215c74c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.106861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.108019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.108672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.109925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.113334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.116113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.129409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7eef48bc4fd41008beffc3dc32de757'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.131198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1041/2057] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:02.135054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.144014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7eef48bc4fd41008beffc3dc32de757'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.153837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5651c-6b96-4599-bc83-06d1fd4aadf6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.162321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.163194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.163709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.164139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] PATH_INFO: `/v3/projects/79b5426c48664784b84885ee264f10ab/users/10971b3576014efcae170bfe252eb0b8/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.166053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7eef48bc4fd41008beffc3dc32de757'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.176183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7eef48bc4fd41008beffc3dc32de757'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.176936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3ff1c6fe-d44c-4b6b-8e78-2d164d473d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7eef48bc4fd41008beffc3dc32de757'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.181841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b5426c48664784b84885ee264f10ab, user_id=10971b3576014efcae170bfe252eb0b8, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.184795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1017/2058] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:01 2022] DELETE /identity/v3/users/e7eef48bc4fd41008beffc3dc32de757 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:02.188752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.189984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.190323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.190803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.214268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1042/2059] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:02 2022] PUT /identity/v3/projects/79b5426c48664784b84885ee264f10ab/users/10971b3576014efcae170bfe252eb0b8/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:02.216770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.218232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:02.218524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.220188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.244698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:02.309618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1018/2060] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:02.320018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.321594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.339981 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b944-74d9-49f7-ba61-c2f94d9b768b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.346735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.347621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.348106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.348567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.349487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.350845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:02.352160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.364481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1043/2061] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:02.370653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.396041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330032-bee1-4527-9086-db2ddc193e8e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.403947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.406624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.407559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.407849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] PATH_INFO: `/v3/projects/79b5426c48664784b84885ee264f10ab/users/10971b3576014efcae170bfe252eb0b8/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.413284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b5426c48664784b84885ee264f10ab, user_id=10971b3576014efcae170bfe252eb0b8, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.415957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae67a5-466a-4a33-8da4-259bcfbd7012 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.423066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1474-acd0-4999-a2c5-178b3cc52575 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.427615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.428275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.428569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.429331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.434463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1044/2062] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:02 2022] PUT /identity/v3/projects/79b5426c48664784b84885ee264f10ab/users/10971b3576014efcae170bfe252eb0b8/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:02.439800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.441263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.444789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.445150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.445432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.452659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1019/2063] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:02.478199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] MFA Rules not processed for user `10971b3576014efcae170bfe252eb0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:02.479462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.594844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1045/2064] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:02.611592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.613455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.684108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8936b-725b-4f5d-9355-60c45470db79 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.750373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.751570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.751920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.752199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.755360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.777566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.779514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.865333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.866810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.870035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27bdf2-54dd-4b98-b20e-aa6250e5a3f6 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.875499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.876258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.876618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.876986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] PATH_INFO: `/v3/projects/052c415fcbd34eebbd5b50af4e10a74f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.878020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1020/2065] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:02.881683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=052c415fcbd34eebbd5b50af4e10a74f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.884881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.904643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:02.946374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052c415fcbd34eebbd5b50af4e10a74f', 'request_id': 'req-bf17cefe-b587-4c12-9614-18e96d075edc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.954277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052c415fcbd34eebbd5b50af4e10a74f', 'request_id': 'req-bf17cefe-b587-4c12-9614-18e96d075edc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.961865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052c415fcbd34eebbd5b50af4e10a74f', 'request_id': 'req-bf17cefe-b587-4c12-9614-18e96d075edc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.986514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] The token cache is being invalidate because project 052c415fcbd34eebbd5b50af4e10a74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:02.986810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.987612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.990766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1046/2066] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:02 2022] DELETE /identity/v3/projects/052c415fcbd34eebbd5b50af4e10a74f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:02.994621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:03.008631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af610ed-c82e-4bc2-928c-ccfe009d0005 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.056937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:03.108036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.110308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.110856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.111495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] PATH_INFO: `/v3/users/15ff53cba0674db4b8d42516e3fc2c69` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.114240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ff53cba0674db4b8d42516e3fc2c69)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.117870 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.190427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ff53cba0674db4b8d42516e3fc2c69'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.201035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ff53cba0674db4b8d42516e3fc2c69'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.218396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ff53cba0674db4b8d42516e3fc2c69'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.228748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ff53cba0674db4b8d42516e3fc2c69'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.230166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ff53cba0674db4b8d42516e3fc2c69'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.237198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1021/2067] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:02 2022] DELETE /identity/v3/users/15ff53cba0674db4b8d42516e3fc2c69 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:03.660450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:03.662049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.678339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17cefe-b587-4c12-9614-18e96d075edc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.762161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.762837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.763110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.763377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] PATH_INFO: `/v3/projects/97e7a229a45149e8a66dc60fb70b5af3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.765469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e7a229a45149e8a66dc60fb70b5af3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.767680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.815955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e7a229a45149e8a66dc60fb70b5af3', 'request_id': 'req-ed688df0-d3d9-4788-bd53-3516ab12bb2b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.822164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e7a229a45149e8a66dc60fb70b5af3', 'request_id': 'req-ed688df0-d3d9-4788-bd53-3516ab12bb2b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.829163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e7a229a45149e8a66dc60fb70b5af3', 'request_id': 'req-ed688df0-d3d9-4788-bd53-3516ab12bb2b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.845305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] The token cache is being invalidate because project 97e7a229a45149e8a66dc60fb70b5af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:03.845595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.846272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:03.847771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1047/2068] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:03 2022] DELETE /identity/v3/projects/97e7a229a45149e8a66dc60fb70b5af3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:03.851893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:03.853389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.868028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51e406-121c-4f8f-a9e1-2f7f9fd6282a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.954188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.955011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.955395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.955772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] PATH_INFO: `/v3/users/cb09dc07110740bc8ec7e9f7db2d8e7f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.957075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb09dc07110740bc8ec7e9f7db2d8e7f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.959128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.023725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb09dc07110740bc8ec7e9f7db2d8e7f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.027913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:04.034854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb09dc07110740bc8ec7e9f7db2d8e7f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.050095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb09dc07110740bc8ec7e9f7db2d8e7f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.056923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb09dc07110740bc8ec7e9f7db2d8e7f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.057668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-5a550e73-5cfd-4082-8d7f-0aeebf05e129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb09dc07110740bc8ec7e9f7db2d8e7f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.064437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1022/2069] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:03 2022] DELETE /identity/v3/users/cb09dc07110740bc8ec7e9f7db2d8e7f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:04.333224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.335293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.350404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688df0-d3d9-4788-bd53-3516ab12bb2b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.386541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:04.412426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.413163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.413443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.413714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] PATH_INFO: `/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.414814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d43400742db4aaa8c3a9fb1c3f548aa)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.416712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.464358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d43400742db4aaa8c3a9fb1c3f548aa', 'request_id': 'req-2783046e-1fc8-4895-8942-d61985fbf408'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.470463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d43400742db4aaa8c3a9fb1c3f548aa', 'request_id': 'req-2783046e-1fc8-4895-8942-d61985fbf408'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.477153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d43400742db4aaa8c3a9fb1c3f548aa', 'request_id': 'req-2783046e-1fc8-4895-8942-d61985fbf408'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.489525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] The token cache is being invalidate because project 7d43400742db4aaa8c3a9fb1c3f548aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:04.489817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.490455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2783046e-1fc8-4895-8942-d61985fbf408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.491995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1048/2070] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:04 2022] DELETE /identity/v3/projects/7d43400742db4aaa8c3a9fb1c3f548aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:04.497014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.497307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.497588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.517509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:04.518716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.582670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1023/2071] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:04.590705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.591000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.591282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.611171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:04.612371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.620188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1049/2072] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:04.624667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.634410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaf37d-c3c5-42ee-a630-b75938f7d0a8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.639044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.640143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.640426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.640699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.641382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:04.641686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.643256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.655556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1024/2073] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:04.659757 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.668707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457c84d-8608-483b-afd7-07440702cdf5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.673335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.673918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.674195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.674462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.675081 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.676985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.706386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1050/2074] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:04.710701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.721339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a428-1d94-4384-b05a-e57c392fa2ce admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.778016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.778798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.779083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.779356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.780562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.782643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.804760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1025/2075] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:04.808699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.819763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e306bf-873e-402e-9a29-5418e5a11aef admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.824286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.824877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.825185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.825456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.826090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.828264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.837941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1051/2076] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:04.841818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.851809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc861c-1f4d-4299-a8ab-0806dbb34384 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.856426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.857032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.857321 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.857596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.858236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.859805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93333717-0a8a-40a2-abd7-5957a3807238 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.870260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1026/2077] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:04.874837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.884672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90af65-894c-4bd1-a022-2718dd52fb2a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.889177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.890391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.890670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.890943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] PATH_INFO: `/v3/projects/79311488c215429192c50198b51eeb10/users/3356913cfe8e40a5b4bb24a720b40771/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.905082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79311488c215429192c50198b51eeb10, user_id=3356913cfe8e40a5b4bb24a720b40771, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.907735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.920218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1052/2078] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:04 2022] PUT /identity/v3/projects/79311488c215429192c50198b51eeb10/users/3356913cfe8e40a5b4bb24a720b40771/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:04.931399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.931692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.931973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.961198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] MFA Rules not processed for user `3356913cfe8e40a5b4bb24a720b40771`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:04.962427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.987347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:05.030711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1027/2079] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:05.042094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:05.043795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.114344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23cd11-ce88-455d-998b-e4c0f56aad90 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:05.118268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:05.118929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:05.119215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:05.119540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:05.130636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:05.132503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:05.143462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1053/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:05.594084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:05.595659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.607603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f084e27-5f6e-406e-8653-6096535e950a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:05.611339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:05.611932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:05.612231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:05.612537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:05.614275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.672291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:05.673747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:05.682466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1028/2081] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:06.229868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:06.236546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:06.242117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f66df-6566-4980-9d9f-12c65e1ee759 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:06.249547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:06.251711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:06.251911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:06.252492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:06.254925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:06.321434 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:06.322946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:06.334086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1054/2082] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:09.832854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:09.834921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:09.911766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013522a-ac62-4ac6-95b1-7dfba0276f41 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:09.916386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:09.917119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:09.917463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:09.917812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] PATH_INFO: `/v3/users/5b9f29fab9e24e98a632416f2160182c` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:09.919027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9f29fab9e24e98a632416f2160182c)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:09.921458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:09.971564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9f29fab9e24e98a632416f2160182c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.979665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9f29fab9e24e98a632416f2160182c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.989521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:09.995847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9f29fab9e24e98a632416f2160182c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.004086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9f29fab9e24e98a632416f2160182c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.004842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea7aae07-ae3b-4549-87ab-682ac43a5049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9f29fab9e24e98a632416f2160182c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.011680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1029/2083] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:09 2022] DELETE /identity/v3/users/5b9f29fab9e24e98a632416f2160182c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:10.194866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.206820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba3718-cee1-4480-ac02-8cfd70aec58e service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.266436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.267143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.267489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.267827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] PATH_INFO: `/v3/projects/3f3531fb48774df8a26ee722f3eb837e` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.270883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3531fb48774df8a26ee722f3eb837e)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.270997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.322929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3531fb48774df8a26ee722f3eb837e', 'request_id': 'req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.328495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3531fb48774df8a26ee722f3eb837e', 'request_id': 'req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.334545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3531fb48774df8a26ee722f3eb837e', 'request_id': 'req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.348004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] The token cache is being invalidate because project 3f3531fb48774df8a26ee722f3eb837e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:10.348285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.348911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fff114c7-0f4c-4ea7-81a6-7dc6c8e65153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.350644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1055/2084] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:10 2022] DELETE /identity/v3/projects/3f3531fb48774df8a26ee722f3eb837e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:10.355383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.355677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.355961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.377705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:10.378927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.449999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1030/2085] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:10.459270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.459562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.459839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.482507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:10.484324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.492710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1056/2086] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:10.497402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.508187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f06fb9-0c6a-408d-94a1-9d3e80b818e8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.514063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.514759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.514999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.515268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.516118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:10.516392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.518098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.530522 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1031/2087] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:10.534716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.544187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36149555-a825-485a-86dc-e13d2426bf73 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.550155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.550767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.551048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.551350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.552119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.554005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.576210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:10.583897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1057/2088] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:10.587959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.598872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456515f-cd24-41b0-a02b-ee1239621a31 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.654957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.655560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.655839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.656117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.656732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.658684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.680334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1032/2089] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:10.684571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.695873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad50e-c2a3-45da-b337-2535db29f4ef admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.700608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.701241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.701518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.701795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.702422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.704035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.714355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1058/2090] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:10.718229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.728001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19f446-d15a-4078-b602-9279d4ee1dd5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.732580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.733724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.734005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.734274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.734900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.736464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff1299-c661-4634-98e4-12356a6c1ddb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.746897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1033/2091] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:10.751770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.761630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2440399-5b9c-488b-91b4-00dd99cba876 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.766766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.767554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.767874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.768226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] PATH_INFO: `/v3/projects/3b7f5b65cb8541f293b269f50d14109d/users/ca8e30e9b9cb499ba07801566cf9fd92/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.783635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7f5b65cb8541f293b269f50d14109d, user_id=ca8e30e9b9cb499ba07801566cf9fd92, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.786819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.800581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1059/2092] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:10 2022] PUT /identity/v3/projects/3b7f5b65cb8541f293b269f50d14109d/users/ca8e30e9b9cb499ba07801566cf9fd92/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:10.812503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.812791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.813100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.843632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] MFA Rules not processed for user `ca8e30e9b9cb499ba07801566cf9fd92`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:10.844863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.885309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:10.916692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1034/2093] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:10.921739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.983698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a210d-f6f1-4a0b-bffd-8cec085925f3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.988226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.988819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.989134 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.989404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.990013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.991935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.017274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1060/2094] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:11.021390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.032761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6cd3-81e7-44f8-b400-965940881794 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.088520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.089165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.089444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.089715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.090357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.092765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.114187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1035/2095] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:11.118091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.128731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b93dce-bce1-442d-8a0e-a81f067314a5 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.133393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.133991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.134275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.134551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.135189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.136874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.150854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1061/2096] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:11.155456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.165853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14729e-713e-4318-aa8a-9b79353f3d13 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.170432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.171081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.171361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.171634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] PATH_INFO: `/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7/users/b9c217ac8f2740649e7be83f452fe78f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.189898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0556df55e63406b9dcd69aa2ec93cb7, user_id=b9c217ac8f2740649e7be83f452fe78f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.192515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.205869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1036/2097] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:11 2022] PUT /identity/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7/users/b9c217ac8f2740649e7be83f452fe78f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:11.209875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.222454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f2135-cc01-43ea-ba5f-94868649557d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.280202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.280808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.281124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.281402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.282031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.283638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94337a-7cff-4a72-9626-ccb57c92cc81 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.293279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1062/2098] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:11.297916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.308443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684d66-93e1-45b4-a2c2-8144cfba6ff0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.313751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.314415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.314698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.314980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] PATH_INFO: `/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7/users/b9c217ac8f2740649e7be83f452fe78f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.316884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0556df55e63406b9dcd69aa2ec93cb7, user_id=b9c217ac8f2740649e7be83f452fe78f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.319810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.332567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1037/2099] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:11 2022] PUT /identity/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7/users/b9c217ac8f2740649e7be83f452fe78f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:11.343779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.344069 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.344348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.372740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.375033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.375161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] MFA Rules not processed for user `b9c217ac8f2740649e7be83f452fe78f`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:11.376232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.403871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:11.463369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1063/2100] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:11.466702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56389718-e7b2-443d-a919-873197da8bbc admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.471634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.472306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.472585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.472887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] PATH_INFO: `/v3/users/33ef9645a7834dfda95b1d616854390a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.474309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.474429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ef9645a7834dfda95b1d616854390a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.475701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.476240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.521831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:11.540887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ef9645a7834dfda95b1d616854390a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.547276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b9a7-9262-4dce-83a9-b2b1d9740788 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.551391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ef9645a7834dfda95b1d616854390a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.551679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.552425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.553041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.553041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.564646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.567597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53404-4a9b-4f8b-8bbb-2ca55b119a55 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.572651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ef9645a7834dfda95b1d616854390a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.577339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1064/2101] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:11.580850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ef9645a7834dfda95b1d616854390a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.581318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ef9645a7834dfda95b1d616854390a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.587820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1038/2102] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:11 2022] DELETE /identity/v3/users/33ef9645a7834dfda95b1d616854390a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:11.594969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.594969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.594969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.625942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:11.627264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.704214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1065/2103] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:11.715359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.716822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.782347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8a4bb-66b4-4e0c-b4f8-e41349871475 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.786202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.787423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.787715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.788065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.798777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.800199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83544d-fd88-49cf-bf5b-715946909f61 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.810067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1039/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:11.826626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.827954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.841703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16160c-4c1a-4b57-b974-0e4c5d74b514 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.847004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.847763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.848040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.848324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] PATH_INFO: `/v3/users/ca8e30e9b9cb499ba07801566cf9fd92` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.849508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8e30e9b9cb499ba07801566cf9fd92)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.851472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.901341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8e30e9b9cb499ba07801566cf9fd92'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.901994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8e30e9b9cb499ba07801566cf9fd92'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.911014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:11.917019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8e30e9b9cb499ba07801566cf9fd92'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.925285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8e30e9b9cb499ba07801566cf9fd92'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.935712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8e30e9b9cb499ba07801566cf9fd92'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.947701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1066/2105] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:11 2022] DELETE /identity/v3/users/ca8e30e9b9cb499ba07801566cf9fd92 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:11.953426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74926163-5ccf-46c2-b302-694474f60882 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.954631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74926163-5ccf-46c2-b302-694474f60882 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.955333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74926163-5ccf-46c2-b302-694474f60882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.983746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-74926163-5ccf-46c2-b302-694474f60882 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:11.985020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-74926163-5ccf-46c2-b302-694474f60882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.050673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1040/2106] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:12.060196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.061712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.111807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74926163-5ccf-46c2-b302-694474f60882 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.113800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-74926163-5ccf-46c2-b302-694474f60882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.126971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74926163-5ccf-46c2-b302-694474f60882 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.132383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.133162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.133693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.134590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e9e1-100e-47f9-ac50-3bb387903220 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.134713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] PATH_INFO: `/v3/projects/e4f2d720a5ef47c995eaec110c676761` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.135303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f2d720a5ef47c995eaec110c676761)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.137479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.138797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.140355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.140763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.141579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.154010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.155570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.166300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1067/2107] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:12.190252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:12.191815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f2d720a5ef47c995eaec110c676761', 'request_id': 'req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.201841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f2d720a5ef47c995eaec110c676761', 'request_id': 'req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.209681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f2d720a5ef47c995eaec110c676761', 'request_id': 'req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.220712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] The token cache is being invalidate because project e4f2d720a5ef47c995eaec110c676761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:12.221107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.221999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.223650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1041/2108] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/projects/e4f2d720a5ef47c995eaec110c676761 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:12.228118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.239197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ca7b2-cd12-4a48-9a3e-75c205736c86 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.299494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.300124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.300406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.300680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] PATH_INFO: `/v3/users/a992cd1260274c69a2cf5691139420c1` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.301867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a992cd1260274c69a2cf5691139420c1)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.303784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.353735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a992cd1260274c69a2cf5691139420c1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.354578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a992cd1260274c69a2cf5691139420c1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.369011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a992cd1260274c69a2cf5691139420c1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.376271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a992cd1260274c69a2cf5691139420c1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.384381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a992cd1260274c69a2cf5691139420c1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.385939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:12.393455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1068/2109] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/users/a992cd1260274c69a2cf5691139420c1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:12.449311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.450961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.476290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc5ac8-74c1-4e22-bfce-929fbc7dcc67 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.550098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.550743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.551027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.551300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] PATH_INFO: `/v3/projects/3b7f5b65cb8541f293b269f50d14109d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.552412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7f5b65cb8541f293b269f50d14109d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.554361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.608427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7f5b65cb8541f293b269f50d14109d', 'request_id': 'req-2d25937e-f779-486a-964a-b028ace789a0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.615589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7f5b65cb8541f293b269f50d14109d', 'request_id': 'req-2d25937e-f779-486a-964a-b028ace789a0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.622042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7f5b65cb8541f293b269f50d14109d', 'request_id': 'req-2d25937e-f779-486a-964a-b028ace789a0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.633490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] The token cache is being invalidate because project 3b7f5b65cb8541f293b269f50d14109d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:12.633814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.634471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.635962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1042/2110] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/projects/3b7f5b65cb8541f293b269f50d14109d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:12.639982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.641496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.656065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1cf7-42d2-41c4-b8af-6c7fdab4dd21 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.703444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.706035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.729300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.730384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.730920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.732226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] PATH_INFO: `/v3/users/b9c217ac8f2740649e7be83f452fe78f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.733742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c217ac8f2740649e7be83f452fe78f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.735930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.792400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c217ac8f2740649e7be83f452fe78f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.793342 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c217ac8f2740649e7be83f452fe78f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.799587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25937e-f779-486a-964a-b028ace789a0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.804832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.805437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.805695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.806048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] PATH_INFO: `/v3/projects/840f194741544f298a5bbe3f2b1c7f13` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.807233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840f194741544f298a5bbe3f2b1c7f13)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.808803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c217ac8f2740649e7be83f452fe78f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.809254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.820551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c217ac8f2740649e7be83f452fe78f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.822085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:12.837687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c217ac8f2740649e7be83f452fe78f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.853490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1069/2111] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/users/b9c217ac8f2740649e7be83f452fe78f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:12.879507 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840f194741544f298a5bbe3f2b1c7f13', 'request_id': 'req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.894052 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840f194741544f298a5bbe3f2b1c7f13', 'request_id': 'req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.901866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840f194741544f298a5bbe3f2b1c7f13', 'request_id': 'req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.916487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] The token cache is being invalidate because project 840f194741544f298a5bbe3f2b1c7f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:12.916880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.917738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.919569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1043/2112] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/projects/840f194741544f298a5bbe3f2b1c7f13 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:12.923761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.925228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.015639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9288c-e6a2-4f1c-8a02-948364675b04 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.055731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.107538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.108517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.108517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.108723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] PATH_INFO: `/v3/users/10971b3576014efcae170bfe252eb0b8` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.110143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=10971b3576014efcae170bfe252eb0b8)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.112506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.174597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10971b3576014efcae170bfe252eb0b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.175500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10971b3576014efcae170bfe252eb0b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.187957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.190412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.195519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10971b3576014efcae170bfe252eb0b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.206415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b17b-2d31-4b15-b7dc-1fe269453fd3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.207041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10971b3576014efcae170bfe252eb0b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.211652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.212457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.212740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.213164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] PATH_INFO: `/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.214604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0556df55e63406b9dcd69aa2ec93cb7)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.217617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.218187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-1d7753b9-f20f-41ca-837f-fb74caa343ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10971b3576014efcae170bfe252eb0b8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.226733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1070/2113] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/users/10971b3576014efcae170bfe252eb0b8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:13.283359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0556df55e63406b9dcd69aa2ec93cb7', 'request_id': 'req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.294552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0556df55e63406b9dcd69aa2ec93cb7', 'request_id': 'req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.302671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0556df55e63406b9dcd69aa2ec93cb7', 'request_id': 'req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.320194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] The token cache is being invalidate because project e0556df55e63406b9dcd69aa2ec93cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:13.320194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.321095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd44840b-e70d-472b-a2fd-3ee5c4ab75c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.323416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1044/2114] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:13 2022] DELETE /identity/v3/projects/e0556df55e63406b9dcd69aa2ec93cb7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:13.328925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.329857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.330147 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.352996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.354274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.455293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1071/2115] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:13.462529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.462823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.463100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.484748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.486154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.497174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1045/2116] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:13.501381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.513166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed00df-05b6-4a5d-9dee-3daabf5515c7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.518476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.519092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.519372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.519646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.520288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:13.520590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.522304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.535482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1072/2117] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:13.539965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.550494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bce61-164b-4c99-9e7d-b1b6d340d99a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.555643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.556289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.556609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.556950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.557745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.559884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.567948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.569982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.589430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230c9ec-acbe-4a46-bf3e-09615948ba5b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.594974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.595622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.595905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.596181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] PATH_INFO: `/v3/projects/79b5426c48664784b84885ee264f10ab` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.597367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b5426c48664784b84885ee264f10ab)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.599389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.601897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1046/2118] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:13.605778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.632531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.655431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b5426c48664784b84885ee264f10ab', 'request_id': 'req-194bd08f-870d-42dc-a0ea-391090cc953d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.661800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.662836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b5426c48664784b84885ee264f10ab', 'request_id': 'req-194bd08f-870d-42dc-a0ea-391090cc953d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.669975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b5426c48664784b84885ee264f10ab', 'request_id': 'req-194bd08f-870d-42dc-a0ea-391090cc953d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.676416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16fef0-9796-4aaf-b99c-35ebc6c3aebb admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.687187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] The token cache is being invalidate because project 79b5426c48664784b84885ee264f10ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:13.687726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.688770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-194bd08f-870d-42dc-a0ea-391090cc953d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.691303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1073/2119] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:13 2022] DELETE /identity/v3/projects/79b5426c48664784b84885ee264f10ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:13.696267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ea83-53e2-4934-a815-39f792630cb0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.696552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ea83-53e2-4934-a815-39f792630cb0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.698150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ea83-53e2-4934-a815-39f792630cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.717350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0478ea83-53e2-4934-a815-39f792630cb0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.718552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0478ea83-53e2-4934-a815-39f792630cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.756627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.757278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.757558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.757831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.758459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.760359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.783220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1047/2120] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:13.785951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1074/2121] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:13.787330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.789211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.793863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.794152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.794425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.801583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67e62-e01c-4848-906f-82295ab209f3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.806206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.806800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.807077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.807350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.807973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.810310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.814358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.815556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.821240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1048/2122] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:13.823838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1075/2123] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:13.825011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.827782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.836021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca990-d381-4707-b034-b9bb42095cfe admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.836858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643f16f-c159-4957-aa9a-846a54b6bb85 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.840820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.841467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.841597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.841752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.842710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.842846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.843040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.843314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.843426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.843949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:13.844251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.846691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.846823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.857808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1049/2124] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:13.858217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1076/2125] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:13.861975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.862136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.872769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7775c-7e87-4114-aced-808785bfde6f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.880343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8379c2-0f2f-414b-a7e2-3a5223a4a872 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.880439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.881280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.881458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.881795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.882433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.884375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.885986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.886822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.887232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.887521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] PATH_INFO: `/v3/projects/06b7cb94796c421cb3c20aeb93ef85fe/users/0541524c5cae4e5e9c10626e6ed746e3/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.906539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b7cb94796c421cb3c20aeb93ef85fe, user_id=0541524c5cae4e5e9c10626e6ed746e3, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.910941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dea722-5f89-4e99-98f0-10d73cec557c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.915439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1077/2126] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:13.919748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.934865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1050/2127] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:13 2022] PUT /identity/v3/projects/06b7cb94796c421cb3c20aeb93ef85fe/users/0541524c5cae4e5e9c10626e6ed746e3/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:13.952385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.952772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.953163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.987199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] MFA Rules not processed for user `0541524c5cae4e5e9c10626e6ed746e3`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.988421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.996698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd81dd-9955-40e9-8d1b-2528778e1139 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.020505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.083011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1051/2128] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:14.085281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.086569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.087490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.089428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.090354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.091581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.095708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.102956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c45ae-e4f2-45ec-8826-12bf4a46e4f9 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.110846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.111994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.112610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.112908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.113780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.116466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.150428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1078/2129] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:14.154791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.162841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1052/2130] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:14.166236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c553-9637-4a8e-b510-7d46eabebcd4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.166742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.178833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.255443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.256061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.256345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.256614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.257918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.261681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.263820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934ce5-3e7f-402d-bc74-c04b1d9db23c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.269874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.270668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.271039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.271482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.272417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.272549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1079/2131] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:14.276154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.277110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.287655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5df86-838a-437b-9983-ecb404504ae7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.292775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.293501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.293936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.294229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.294851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.297091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.302325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1053/2132] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:14.306002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.313397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1080/2133] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:14.314582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.316468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7d4fd-f042-4ef6-a7ce-84992e40b891 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.321369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.322017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.322298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.322573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.323204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.324789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.324905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aeb96-f400-4dc1-9c71-de961de9bfaf admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.330192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.330838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.331250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.331569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] PATH_INFO: `/v3/projects/c761c2cc031f4bd6b6d652e88cff6578/users/29c72f42695a4f5e868e1ffe57332fcf/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.335339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1054/2134] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:14.340112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.347248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c761c2cc031f4bd6b6d652e88cff6578, user_id=29c72f42695a4f5e868e1ffe57332fcf, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.349967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095fdef-4d8c-4a8f-9a69-ee00ea0d72ce admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.350622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33b32-958e-4064-83c3-c7822c7e7c55 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.354617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.355335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.355660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.355986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] PATH_INFO: `/v3/projects/5424d39d657b4bccad54c878b5e284e9/users/cfb5054ba4204e9fb078c0b86082f9e4/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.365750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1081/2135] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:14 2022] PUT /identity/v3/projects/c761c2cc031f4bd6b6d652e88cff6578/users/29c72f42695a4f5e868e1ffe57332fcf/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:14.377173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.377462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.377741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.380372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5424d39d657b4bccad54c878b5e284e9, user_id=cfb5054ba4204e9fb078c0b86082f9e4, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.383050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.396280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1055/2136] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:14 2022] PUT /identity/v3/projects/5424d39d657b4bccad54c878b5e284e9/users/cfb5054ba4204e9fb078c0b86082f9e4/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:14.400058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.407685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] MFA Rules not processed for user `29c72f42695a4f5e868e1ffe57332fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.408986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.460714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778d3f-dd0d-4d67-9920-e16bb4029060 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.466012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.466636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.466987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.467269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.468061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.469959 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.475789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.481130 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1082/2137] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:14.483700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1056/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:14.487556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.492322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.494207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.502054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05320cc6-5d0b-4e6b-987c-715a093a6653 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.506878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.507541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.507828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.508102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] PATH_INFO: `/v3/projects/5424d39d657b4bccad54c878b5e284e9/users/cfb5054ba4204e9fb078c0b86082f9e4/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.510090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5424d39d657b4bccad54c878b5e284e9, user_id=cfb5054ba4204e9fb078c0b86082f9e4, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.512758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64afcd7-10d1-47c5-b344-ecfab43cb9bf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.523383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1083/2139] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:14 2022] PUT /identity/v3/projects/5424d39d657b4bccad54c878b5e284e9/users/cfb5054ba4204e9fb078c0b86082f9e4/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:14.533352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.533725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.534003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.556433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28813e-b0e5-403a-83a8-4a7dd4b50ba9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.563890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] MFA Rules not processed for user `cfb5054ba4204e9fb078c0b86082f9e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:14.565195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.568665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:14.607990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.608605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.608890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.609430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.636422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1084/2140] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:14.648142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.649496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.668088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.669663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.678653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1057/2141] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:14.706876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a431e9-4fcf-4a7e-a35b-72d4bebc69bc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.710535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.711566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.711790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.712563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.721875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.723305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d2f79-965e-47b6-b4b9-877b5b256721 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.733573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1085/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:15.132748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.199656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361f093-d9a2-4f23-b323-36826b49e3a5 service glance] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.205127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.205756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.206040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.206314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] PATH_INFO: `/v3/users/0541524c5cae4e5e9c10626e6ed746e3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.207430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0541524c5cae4e5e9c10626e6ed746e3)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.209377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.258107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0541524c5cae4e5e9c10626e6ed746e3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.267491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0541524c5cae4e5e9c10626e6ed746e3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.287145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0541524c5cae4e5e9c10626e6ed746e3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.288601 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.301114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0541524c5cae4e5e9c10626e6ed746e3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.302626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0541524c5cae4e5e9c10626e6ed746e3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.309829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1058/2143] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:15 2022] DELETE /identity/v3/users/0541524c5cae4e5e9c10626e6ed746e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:15.314561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.314851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.315129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.334989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:15.336250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.400404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1086/2144] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:15.411172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.412819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.472893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b72bb8-4271-4d17-86cb-099ec6caa2ee admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.476573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.477215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.477498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.477773 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.486599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.488017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.497578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1059/2145] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:15.766504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.767974 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.780114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73efa0-8189-4ef5-b885-b544e6e77e69 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.784720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.785383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.785666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.785943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] PATH_INFO: `/v3/projects/06b7cb94796c421cb3c20aeb93ef85fe` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.787067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b7cb94796c421cb3c20aeb93ef85fe)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.789055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.801191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.838449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b7cb94796c421cb3c20aeb93ef85fe', 'request_id': 'req-2449e71a-a384-401d-8fb0-500dd42fbdbc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.843984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b7cb94796c421cb3c20aeb93ef85fe', 'request_id': 'req-2449e71a-a384-401d-8fb0-500dd42fbdbc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.850078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b7cb94796c421cb3c20aeb93ef85fe', 'request_id': 'req-2449e71a-a384-401d-8fb0-500dd42fbdbc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.863484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] The token cache is being invalidate because project 06b7cb94796c421cb3c20aeb93ef85fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:15.863775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.864398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.865890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1087/2146] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:15 2022] DELETE /identity/v3/projects/06b7cb94796c421cb3c20aeb93ef85fe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:15.869988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.882560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67f45c-6d43-4208-a8cc-8f3f39f4cd43 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.939329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.939946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.940224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.940492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] PATH_INFO: `/v3/users/cfb5054ba4204e9fb078c0b86082f9e4` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.941604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb5054ba4204e9fb078c0b86082f9e4)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.943748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.988183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb5054ba4204e9fb078c0b86082f9e4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.995980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb5054ba4204e9fb078c0b86082f9e4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.007977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb5054ba4204e9fb078c0b86082f9e4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.015202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb5054ba4204e9fb078c0b86082f9e4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.015783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe5ee4f1-fae0-41fa-a2fc-25826caf50a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb5054ba4204e9fb078c0b86082f9e4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.021496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1060/2147] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:15 2022] DELETE /identity/v3/users/cfb5054ba4204e9fb078c0b86082f9e4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.292182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.293633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.307089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449e71a-a384-401d-8fb0-500dd42fbdbc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.369063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.369698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.370011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.370284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] PATH_INFO: `/v3/projects/5424d39d657b4bccad54c878b5e284e9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.371406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5424d39d657b4bccad54c878b5e284e9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.373376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.422247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5424d39d657b4bccad54c878b5e284e9', 'request_id': 'req-b178dceb-36be-414a-896d-475a39bbc12c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.427902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5424d39d657b4bccad54c878b5e284e9', 'request_id': 'req-b178dceb-36be-414a-896d-475a39bbc12c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.433946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5424d39d657b4bccad54c878b5e284e9', 'request_id': 'req-b178dceb-36be-414a-896d-475a39bbc12c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.447133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] The token cache is being invalidate because project 5424d39d657b4bccad54c878b5e284e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:16.447421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.448036 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-b178dceb-36be-414a-896d-475a39bbc12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:16.449567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1088/2148] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:16 2022] DELETE /identity/v3/projects/5424d39d657b4bccad54c878b5e284e9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.455002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.455479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.455581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.475615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.476762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.483199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:16.539953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1061/2149] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:16.548067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.548437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.548784 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.569483 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.571312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.580999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1089/2150] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:16.585017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.594968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f991f-9bfb-43c2-b27e-73593e144bd3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.600013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.600625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.600901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.601244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.601863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:16.602163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.603742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.616864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1062/2151] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:16.621493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.634213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5675a-2b16-4c75-bf73-5444f014e103 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.638952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.639557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.639837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.640113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.640739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.642686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.666225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:16.680153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1090/2152] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:16.681250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.692686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc29ee-4628-4119-8671-6042da363e13 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.749449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.750058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.750343 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.750622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.751245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.753178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.774626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1063/2153] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.778849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.788926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89363951-4993-4104-8146-d1a440b57a92 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.793769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.794322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.794605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.794881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.796023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.797788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.807692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1091/2154] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.812018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.822092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703ebeb-afdf-417f-98d7-fec1a1b51434 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.826698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.827294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.827602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.827884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.828523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.830141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cacd7-911c-4a6d-afab-97c8a340460d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.840563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1064/2155] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:16.845163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.854938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c012d-93ac-4c8c-a439-b7f40222ef60 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.860224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.860864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.861184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.861462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] PATH_INFO: `/v3/projects/d6aac641c5f74022a415127bf95ff1f0/users/82111ad2125749ba9df012f9a502deea/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.876197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6aac641c5f74022a415127bf95ff1f0, user_id=82111ad2125749ba9df012f9a502deea, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.878904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.891720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1092/2156] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:16 2022] PUT /identity/v3/projects/d6aac641c5f74022a415127bf95ff1f0/users/82111ad2125749ba9df012f9a502deea/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:16.903272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.903563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.903868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.932367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] MFA Rules not processed for user `82111ad2125749ba9df012f9a502deea`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.933621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.996011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1065/2157] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.000492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.060269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c380-0997-414a-84f2-35cf0d99b6bd admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.064734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.065349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.065628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.065896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.066505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.068663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.097035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1093/2158] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:17.101085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.112201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c9407-e2e3-42ea-af87-17eed7ea1666 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.115742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.168379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.169013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.169299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.169580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.170918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.172820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.193671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1066/2159] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:17.197808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.208892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7cf1e-e7b7-49a3-a65e-33cc8212bb72 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.213778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.214405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.214688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.214965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.215663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.217880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.228696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1094/2160] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:17.233188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.243504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2803d-125d-4150-bab7-b8ae686ed1f2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.248435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.249228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.249579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.249924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] PATH_INFO: `/v3/projects/d274fb4d834b447cb560a2981e771c6b/users/774781c8905b499cbfe3909c50eadeae/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.267362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d274fb4d834b447cb560a2981e771c6b, user_id=774781c8905b499cbfe3909c50eadeae, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.270109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.284804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1067/2161] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:17 2022] PUT /identity/v3/projects/d274fb4d834b447cb560a2981e771c6b/users/774781c8905b499cbfe3909c50eadeae/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.289059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.301986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd86cfd-49bd-4e4e-b110-cfc1b1839934 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.359837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.360371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.360655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.360932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.361602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.363205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb36f37-76e4-4187-a2a5-bb0f0e4f3db5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.373076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1095/2162] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:17.377087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.387834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2447f-3fff-4093-87a0-34ced46bd513 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.393616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.394314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.394621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.394897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] PATH_INFO: `/v3/projects/d274fb4d834b447cb560a2981e771c6b/users/774781c8905b499cbfe3909c50eadeae/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.396802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d274fb4d834b447cb560a2981e771c6b, user_id=774781c8905b499cbfe3909c50eadeae, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.399451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.410557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1068/2163] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:17 2022] PUT /identity/v3/projects/d274fb4d834b447cb560a2981e771c6b/users/774781c8905b499cbfe3909c50eadeae/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:17.420557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.420846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.421154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.448223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] MFA Rules not processed for user `774781c8905b499cbfe3909c50eadeae`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.449465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.473461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.517007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1096/2164] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.528251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.530618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.589918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562928-92dd-4a66-8a89-2c4985f9d534 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.593627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.594261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.594545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.594823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.603803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.605254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dc48e-4218-474e-aedb-333dad14ff60 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.614795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1069/2165] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.050936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.113977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27156b63-83e6-461a-9437-3817ed307cb4 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.118476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.119100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.119382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.119653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] PATH_INFO: `/v3/users/82111ad2125749ba9df012f9a502deea` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.120760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=82111ad2125749ba9df012f9a502deea)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.122722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.167415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82111ad2125749ba9df012f9a502deea'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.167990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82111ad2125749ba9df012f9a502deea'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.179628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82111ad2125749ba9df012f9a502deea'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.187270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82111ad2125749ba9df012f9a502deea'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.195613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82111ad2125749ba9df012f9a502deea'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.202111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1097/2166] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:18 2022] DELETE /identity/v3/users/82111ad2125749ba9df012f9a502deea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:18.206570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.206871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.207139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.227687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:18.228862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.290854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1070/2167] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:18.300634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.302101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.345209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:18.361162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa963bb-f86f-491d-8e95-d781df5641ff admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.364807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.365449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.365729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.366003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.375711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.377193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.386823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1098/2168] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:18.656597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.658190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.665412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:18.671661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292ecb-6a7f-4206-9718-f7ed5d6105f2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.676858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.677558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.677839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.678135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] PATH_INFO: `/v3/projects/d6aac641c5f74022a415127bf95ff1f0` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.679257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6aac641c5f74022a415127bf95ff1f0)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.681163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.739241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6aac641c5f74022a415127bf95ff1f0', 'request_id': 'req-870ff055-b5eb-4e87-a827-d532c5ef2465'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.751382 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6aac641c5f74022a415127bf95ff1f0', 'request_id': 'req-870ff055-b5eb-4e87-a827-d532c5ef2465'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.765779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6aac641c5f74022a415127bf95ff1f0', 'request_id': 'req-870ff055-b5eb-4e87-a827-d532c5ef2465'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.781708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] The token cache is being invalidate because project d6aac641c5f74022a415127bf95ff1f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:18.782042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.782871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.784802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1071/2169] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:18 2022] DELETE /identity/v3/projects/d6aac641c5f74022a415127bf95ff1f0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:18.789366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.802189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68863ac-8bb7-427d-ae44-c855f7a76701 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.866607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.867379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.867658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.868055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] PATH_INFO: `/v3/users/774781c8905b499cbfe3909c50eadeae` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.869223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] RBAC: Authorizing `identity:delete_user(user_id=774781c8905b499cbfe3909c50eadeae)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.871119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.930579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774781c8905b499cbfe3909c50eadeae'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.931274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774781c8905b499cbfe3909c50eadeae'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.945719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774781c8905b499cbfe3909c50eadeae'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.952870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774781c8905b499cbfe3909c50eadeae'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.960481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d27289ca-e266-4da0-b4a0-51479462703e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774781c8905b499cbfe3909c50eadeae'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.969049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1099/2170] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:18 2022] DELETE /identity/v3/users/774781c8905b499cbfe3909c50eadeae => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:19.246543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.248528 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.261539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ff055-b5eb-4e87-a827-d532c5ef2465 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.332314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.333370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.333836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.334297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] PATH_INFO: `/v3/projects/d274fb4d834b447cb560a2981e771c6b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.336119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d274fb4d834b447cb560a2981e771c6b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.339501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.397537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd274fb4d834b447cb560a2981e771c6b', 'request_id': 'req-495257d3-d37c-4495-acc3-1244cad807dc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.404836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd274fb4d834b447cb560a2981e771c6b', 'request_id': 'req-495257d3-d37c-4495-acc3-1244cad807dc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.411241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd274fb4d834b447cb560a2981e771c6b', 'request_id': 'req-495257d3-d37c-4495-acc3-1244cad807dc'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.421661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] The token cache is being invalidate because project d274fb4d834b447cb560a2981e771c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:19.421951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.422586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-495257d3-d37c-4495-acc3-1244cad807dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.424078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1072/2171] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:19 2022] DELETE /identity/v3/projects/d274fb4d834b447cb560a2981e771c6b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:19.428979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.429455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.429553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.449714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:19.450854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.517478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1100/2172] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:19.525326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.525617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.525893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.546393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:19.547620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.555423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1073/2173] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:19.559668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.564931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:19.571291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dc93c-fad3-4c3c-b506-ae32c67c14df None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.577800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.578583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.579033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.579499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.580145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:19.580451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.582082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.593739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1101/2174] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:19.598050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.607872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6bdf-abab-4ba3-8494-88aee87ce341 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.612498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.613153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.613437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.613716 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.614354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.616261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.621782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:19.645651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1074/2175] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:19.649814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.659598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff5d5a-d0d1-4aae-88b2-217d93a2ae9b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.716668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.717198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.717482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.717859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.718552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.720723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.744096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1102/2176] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:19.748759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.759241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06723575-e9da-4c33-8fd2-5e9786f5760c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.764645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.765297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.765581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.765858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.766536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.768172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.778735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1075/2177] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:19.782484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.792494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3171a3-2a98-4b35-956e-c5b45f10ab1d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.797795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.798371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.798652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.798926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.799560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.801241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55212550-1b92-4160-8136-9d8b367aeb5c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.817015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1103/2178] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:19.818508 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.829363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95e70-bfd3-4ada-a0c0-8f5a13e38d9b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.834205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.834881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.835165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.835442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] PATH_INFO: `/v3/projects/dabd2a011803437b9a38b9a79bafb17e/users/498140cc81dd44c6ba005779601f05b4/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.850654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabd2a011803437b9a38b9a79bafb17e, user_id=498140cc81dd44c6ba005779601f05b4, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.853324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.866894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1076/2179] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:19 2022] PUT /identity/v3/projects/dabd2a011803437b9a38b9a79bafb17e/users/498140cc81dd44c6ba005779601f05b4/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:19.878463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.878965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.879248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.910481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] MFA Rules not processed for user `498140cc81dd44c6ba005779601f05b4`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:19.913163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.980649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1104/2180] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:19.986018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.051448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbb1c1-35c3-46a5-ab45-85b30518a84a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.056186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.056799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.057123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.057403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.058032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.059989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.095822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1077/2181] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/projects => generated 357 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:20.099774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.111524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a9d38-64f2-4344-a8bc-57ea51c5ddb7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.166964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:20.174457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.175294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.175753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.176094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.176741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.178747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.203881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1105/2182] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:20 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:20.205268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.215591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5f16-b1fb-4cd9-b274-7182b21019a3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.220793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.221423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.221706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.221980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.222612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.224190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.234621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1078/2183] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:20.238476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.248275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba91cdc-6c77-4dab-bbd2-3df5ecc4c576 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.252866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.253609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.253925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.254197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] PATH_INFO: `/v3/projects/531ecc80734a408e8f64a377bbbee8f2/users/be07ca12bd00422e93556267521f602e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.270245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531ecc80734a408e8f64a377bbbee8f2, user_id=be07ca12bd00422e93556267521f602e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.272920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.285760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1106/2184] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:20 2022] PUT /identity/v3/projects/531ecc80734a408e8f64a377bbbee8f2/users/be07ca12bd00422e93556267521f602e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:20.289855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.300570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a24b7-e611-4bc8-8de2-c12bec76885d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.358920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.359524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.359807 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.360081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.360831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.362623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7758fc0-e9bb-4e07-930b-848396d6cdec admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.373598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1079/2185] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:20.378325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.389824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dac36-9d48-44a9-b4d3-7ade3bf29ee6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.395128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.395639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.395923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.396201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] PATH_INFO: `/v3/projects/531ecc80734a408e8f64a377bbbee8f2/users/be07ca12bd00422e93556267521f602e/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.398305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531ecc80734a408e8f64a377bbbee8f2, user_id=be07ca12bd00422e93556267521f602e, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.401127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.411503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1107/2186] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:20 2022] PUT /identity/v3/projects/531ecc80734a408e8f64a377bbbee8f2/users/be07ca12bd00422e93556267521f602e/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:20.426738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.427029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.427311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.455576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] MFA Rules not processed for user `be07ca12bd00422e93556267521f602e`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:20.456785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:20.465836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:20.526613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1080/2187] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:20.539346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.541117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:20.600157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d404-02cd-4e79-bd10-9ce64a452828 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.603854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.604607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.604891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.605261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.615684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.617167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.626557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1108/2188] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:20.701900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.712366 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf688e0-4ad5-4865-a149-1d821cad09ab None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.715999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.717126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.717424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.717699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.774256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.775734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694bb8c-6327-43fd-9d48-7b4c652fd1ac service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.785441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1081/2189] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:30.622339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=44,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:30.781275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=52 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:46.750392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.825665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13e0fc-376a-4ebe-8f40-083aa722d430 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.830246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.830876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.831158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.831432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] PATH_INFO: `/v3/users/498140cc81dd44c6ba005779601f05b4` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.832546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498140cc81dd44c6ba005779601f05b4)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.834510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.882313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498140cc81dd44c6ba005779601f05b4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:46.882966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498140cc81dd44c6ba005779601f05b4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:46.894774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498140cc81dd44c6ba005779601f05b4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:46.901631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498140cc81dd44c6ba005779601f05b4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:46.909508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498140cc81dd44c6ba005779601f05b4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:46.915738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1109/2190] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:46 2022] DELETE /identity/v3/users/498140cc81dd44c6ba005779601f05b4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:46.922155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.922768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.922768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.942449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:46.943916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.003179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1082/2191] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:47.013753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.015531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.081278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea426aa5-0a0a-42d5-9b85-bad0918ed6c3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.085751 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.086363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.086653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.086923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.097997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.099546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.112279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1110/2192] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:47.383906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.385430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.401516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94889-90bc-4a3f-9453-740ccd58f6c7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.406633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.407805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.407805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.408000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] PATH_INFO: `/v3/projects/dabd2a011803437b9a38b9a79bafb17e` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.409191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dabd2a011803437b9a38b9a79bafb17e)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.411423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.463736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dabd2a011803437b9a38b9a79bafb17e', 'request_id': 'req-f59d80a1-4086-4550-8c7f-339a9f080b18'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.472723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dabd2a011803437b9a38b9a79bafb17e', 'request_id': 'req-f59d80a1-4086-4550-8c7f-339a9f080b18'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.479449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dabd2a011803437b9a38b9a79bafb17e', 'request_id': 'req-f59d80a1-4086-4550-8c7f-339a9f080b18'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.491150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] The token cache is being invalidate because project dabd2a011803437b9a38b9a79bafb17e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:47.491530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.492065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.493682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1083/2193] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:47 2022] DELETE /identity/v3/projects/dabd2a011803437b9a38b9a79bafb17e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:47.498068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.508178 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e34237-9894-46bd-b575-efe89cd41d05 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.529055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:47.575695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.577827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.586309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.587246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.587624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.588087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] PATH_INFO: `/v3/users/be07ca12bd00422e93556267521f602e` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.589328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be07ca12bd00422e93556267521f602e)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.591934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.652320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be07ca12bd00422e93556267521f602e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.653169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be07ca12bd00422e93556267521f602e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.666235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be07ca12bd00422e93556267521f602e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.670858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d80a1-4086-4550-8c7f-339a9f080b18 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.673820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be07ca12bd00422e93556267521f602e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.677779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.678587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.679122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.679728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] PATH_INFO: `/v3/users/58567ef3bb864a72a7bc7014d9ca25c4` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.681393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=58567ef3bb864a72a7bc7014d9ca25c4)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.682227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f4812be-eab8-46e3-8894-13c0d54edb6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be07ca12bd00422e93556267521f602e'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.683732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.690255 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1111/2194] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:47 2022] DELETE /identity/v3/users/be07ca12bd00422e93556267521f602e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:47.718756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:47.744057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58567ef3bb864a72a7bc7014d9ca25c4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.755811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58567ef3bb864a72a7bc7014d9ca25c4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.774151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58567ef3bb864a72a7bc7014d9ca25c4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.784503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58567ef3bb864a72a7bc7014d9ca25c4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.785994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58567ef3bb864a72a7bc7014d9ca25c4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:47.808079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1084/2195] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:47 2022] DELETE /identity/v3/users/58567ef3bb864a72a7bc7014d9ca25c4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:47.812907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.813473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.813735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.838478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:47.839939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.928094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1112/2196] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:47.937829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.939346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.020883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.022526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.023948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da2bf4-419f-4a56-a99c-7ed05cb1e2ab admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.033877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.034521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.034804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.035076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.036435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ce4cc-83ca-47cb-a8c4-1e02ebd9b46f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.048222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.049074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.049461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.049756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.049907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] PATH_INFO: `/v3/projects/531ecc80734a408e8f64a377bbbee8f2` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.051325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621b422-7043-4b32-b869-c13cc60f9fba service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.051890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=531ecc80734a408e8f64a377bbbee8f2)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.054157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.065385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1085/2197] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:48.127199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:48.139770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531ecc80734a408e8f64a377bbbee8f2', 'request_id': 'req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.153669 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531ecc80734a408e8f64a377bbbee8f2', 'request_id': 'req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.163192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531ecc80734a408e8f64a377bbbee8f2', 'request_id': 'req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.180736 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] The token cache is being invalidate because project 531ecc80734a408e8f64a377bbbee8f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:48.181189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.181973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9fe54eec-1b3d-4581-ac3d-fb21cbbd34aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.188184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1113/2198] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:48 2022] DELETE /identity/v3/projects/531ecc80734a408e8f64a377bbbee8f2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:48.194895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.195436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.195956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.218598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:48.219875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.307602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1086/2199] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:48.317889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.318267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.318574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.353492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:48.355015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.363844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1114/2200] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:48.368078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.378713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb84c0a-07b7-4699-bb5a-7300c27c1e81 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.384122 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.384796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.385118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.385493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.386201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:48.386509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.388702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.401909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.404072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.406298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1087/2201] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:48.407515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.428673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b44d0-45f5-44ca-a145-642802d658b4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.435109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.435630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81220e-c1d6-43ed-b7b6-f29b578ff688 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.435932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.436324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.436750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.437502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.439478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.445124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.445863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.446284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.447172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] PATH_INFO: `/v3/projects/ddb141ed4e1d4c5d8e41bd42712ed6e0` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.448883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb141ed4e1d4c5d8e41bd42712ed6e0)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.451388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.476093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1088/2202] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:48.480678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.501133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:48.521655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb141ed4e1d4c5d8e41bd42712ed6e0', 'request_id': 'req-0be179d9-8718-4648-bb6c-fc4dbb111e80'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.532373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb141ed4e1d4c5d8e41bd42712ed6e0', 'request_id': 'req-0be179d9-8718-4648-bb6c-fc4dbb111e80'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.541152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb141ed4e1d4c5d8e41bd42712ed6e0', 'request_id': 'req-0be179d9-8718-4648-bb6c-fc4dbb111e80'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.562318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] The token cache is being invalidate because project ddb141ed4e1d4c5d8e41bd42712ed6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:48.562824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.571857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:48.577845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e424ac9-2958-42c5-88a6-78ca2100e781 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.585489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1115/2203] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:48 2022] DELETE /identity/v3/projects/ddb141ed4e1d4c5d8e41bd42712ed6e0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:48.662266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.662897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.663175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.663447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.664087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.666072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.698440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1089/2204] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/users => generated 284 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:48.700401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.702746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.720470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be179d9-8718-4648-bb6c-fc4dbb111e80 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.730613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.730613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.731879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.732239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.733153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.734973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.748281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1116/2205] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:48.752844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.764287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5b641-5f87-43a8-aa01-54c96a56764a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.785279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.786330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.786745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.787992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.788662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.790319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db3ea-5a98-47c9-9014-4e771c6c06e6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.804994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1090/2206] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:48.806394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.817181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9fa19-b10d-4940-be4f-006527f8b9fa admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.822975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.823640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.823922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.824197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] PATH_INFO: `/v3/projects/79d00034793a4bd0ae63ed61848724b7/users/b77e63b1e29247109149538740f9aa67/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.841196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d00034793a4bd0ae63ed61848724b7, user_id=b77e63b1e29247109149538740f9aa67, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.843947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.857985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1117/2207] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:48 2022] PUT /identity/v3/projects/79d00034793a4bd0ae63ed61848724b7/users/b77e63b1e29247109149538740f9aa67/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:48.868333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.868624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.868927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.896922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] MFA Rules not processed for user `b77e63b1e29247109149538740f9aa67`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:48.898409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.966985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1091/2208] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:48.971429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.031563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876fad6-1457-46b0-90d5-0a8b3c7dd54a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.036158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.036760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.037084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.037353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.037990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.039873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.050697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:49.068633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1118/2209] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:49.072778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.084059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e87e87-a080-4c1c-a016-d604d693ca76 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.107969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:49.140977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.141612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.141893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.142169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.142791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.144690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.169298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1092/2210] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:49 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:49.170449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.180689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710971a7-9943-475e-9f37-c823a5420429 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.185999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.186585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.186858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.187126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.187750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.189419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.199233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1119/2211] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:49.203703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.213435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019f0f6-a5b3-4f14-a706-81c9df1e815f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.218516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.219157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.219435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.219705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] PATH_INFO: `/v3/projects/2abb2ac09cd0472f9b944c28b71a9976/users/131df8ef0a51440797b7a1e6fa77434a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.234550 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abb2ac09cd0472f9b944c28b71a9976, user_id=131df8ef0a51440797b7a1e6fa77434a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.237375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.249937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1093/2212] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:49 2022] PUT /identity/v3/projects/2abb2ac09cd0472f9b944c28b71a9976/users/131df8ef0a51440797b7a1e6fa77434a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:49.253954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.263834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9912f6-fd89-40b2-b1c7-b52e05f737e6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.319127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.319657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.319943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.320229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.320842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.322476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa07d5-64b2-443b-9e47-1ef245664ab6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.331725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1120/2213] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:49.335982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.347499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed534e7-268f-4ccc-b94e-99cbde81921e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.352279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.352950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.353268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.353547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] PATH_INFO: `/v3/projects/2abb2ac09cd0472f9b944c28b71a9976/users/131df8ef0a51440797b7a1e6fa77434a/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.355460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abb2ac09cd0472f9b944c28b71a9976, user_id=131df8ef0a51440797b7a1e6fa77434a, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.358217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.369089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1094/2214] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:49 2022] PUT /identity/v3/projects/2abb2ac09cd0472f9b944c28b71a9976/users/131df8ef0a51440797b7a1e6fa77434a/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:49.380592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.380882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.381199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.410648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] MFA Rules not processed for user `131df8ef0a51440797b7a1e6fa77434a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:49.412016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.483067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1121/2215] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:49.494391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.496178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.560992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ada014-876b-4dd8-97aa-3d92e8cf4987 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.564665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.565313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.566205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.566491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.576251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.577699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.588467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1095/2216] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:49.650401 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.651851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.664496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380229b-b7f9-4608-91a6-9ae655d5ef13 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.668072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.668699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.669010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.669287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.728046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.729576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a33fcb-9a90-4d77-8024-2cfe4fb4718d service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.734115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:49.738891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1122/2217] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:53.051604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:53.114322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dce4da-9b6d-486d-b9d1-0b7bfeafd028 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:53.118820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:53.119545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.119833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.120109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] PATH_INFO: `/v3/users/b77e63b1e29247109149538740f9aa67` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.121849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77e63b1e29247109149538740f9aa67)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:53.123718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:53.148049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:53.174867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77e63b1e29247109149538740f9aa67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.181974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77e63b1e29247109149538740f9aa67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.194985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77e63b1e29247109149538740f9aa67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.202580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77e63b1e29247109149538740f9aa67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.203309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77e63b1e29247109149538740f9aa67'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.209313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1096/2218] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:53 2022] DELETE /identity/v3/users/b77e63b1e29247109149538740f9aa67 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:53.213798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.214256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.214375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.233389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:53.234797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:53.297480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1123/2219] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:53.306904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:53.308319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:53.367805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808f75-7441-40e6-948c-c265d9120a50 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:53.372192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:53.372830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.373143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.373413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.383131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:53.384540 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:53.394153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1097/2220] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:53.835993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:53.837796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:53.855615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93bf5f-2369-4994-8058-299d93a93a13 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:53.861553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:53.862181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.862460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.862724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] PATH_INFO: `/v3/projects/79d00034793a4bd0ae63ed61848724b7` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.863824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d00034793a4bd0ae63ed61848724b7)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:53.865778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:53.924951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d00034793a4bd0ae63ed61848724b7', 'request_id': 'req-28f88cd5-7047-453c-865c-60990426c1f1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.930595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d00034793a4bd0ae63ed61848724b7', 'request_id': 'req-28f88cd5-7047-453c-865c-60990426c1f1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.941072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d00034793a4bd0ae63ed61848724b7', 'request_id': 'req-28f88cd5-7047-453c-865c-60990426c1f1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.954903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] The token cache is being invalidate because project 79d00034793a4bd0ae63ed61848724b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:53.955196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.955820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.957573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1124/2221] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:53 2022] DELETE /identity/v3/projects/79d00034793a4bd0ae63ed61848724b7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:53.961449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:53.971619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a972b-8fc3-4f29-b9d0-f019bcd8c8ca service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.032821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.033484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.033764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.034037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] PATH_INFO: `/v3/users/131df8ef0a51440797b7a1e6fa77434a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.035185 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131df8ef0a51440797b7a1e6fa77434a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.037119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.097463 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:54.099735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131df8ef0a51440797b7a1e6fa77434a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.108660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131df8ef0a51440797b7a1e6fa77434a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.124262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131df8ef0a51440797b7a1e6fa77434a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.134848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131df8ef0a51440797b7a1e6fa77434a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.136500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-33e63e4a-0d1d-4d85-a788-5791e4a51d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131df8ef0a51440797b7a1e6fa77434a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.147083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1098/2222] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:53 2022] DELETE /identity/v3/users/131df8ef0a51440797b7a1e6fa77434a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:54.501816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.503273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.518318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f88cd5-7047-453c-865c-60990426c1f1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.577236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.577788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.578071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.578336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] PATH_INFO: `/v3/projects/2abb2ac09cd0472f9b944c28b71a9976` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.579443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abb2ac09cd0472f9b944c28b71a9976)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.581381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.611933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:54.631555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abb2ac09cd0472f9b944c28b71a9976', 'request_id': 'req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.637703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abb2ac09cd0472f9b944c28b71a9976', 'request_id': 'req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.644261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abb2ac09cd0472f9b944c28b71a9976', 'request_id': 'req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.657446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] The token cache is being invalidate because project 2abb2ac09cd0472f9b944c28b71a9976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:54.657886 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.658528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-47510d17-a0c4-448c-b6c2-c9a4e1c511bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:54.660051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1125/2223] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:54 2022] DELETE /identity/v3/projects/2abb2ac09cd0472f9b944c28b71a9976 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:54.664939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.665262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.665545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.689987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:54.691309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.750984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1099/2224] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:54.758943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.759256 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.759533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.779187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:54.780400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.788832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1126/2225] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:54.793830 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.803530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b50fb-a729-4b58-91d9-31b68364b608 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.808050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.808646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.809531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.809815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.810488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:54.810792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.812373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.824025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1100/2226] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:54.828508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.838938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e41e-a970-4921-b320-6ed266b18c48 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.843722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.844331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.844611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.844880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.847804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.849878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.884446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1127/2227] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:54.894957 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.905457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110e2ba-3e5e-4bca-964e-29ed83c10e86 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.962738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.963356 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.963643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.963918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.964547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.966532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.988516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1101/2228] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:54.992897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.003972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558be32-a732-4eb4-9c13-b9dce826fd1e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.008573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.009206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.009488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.009767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.010426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.012088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.021940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1128/2229] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.026603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.036648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6722f9a-ef40-4354-bc7e-8f9839cab59f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.041412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.042016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.042297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.042572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.043709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.045329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a469071-c654-44b8-9d08-9e20cc9e76b2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.055946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1102/2230] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.060781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.071226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07006d-78c7-4ce8-b9a8-c98e2e35ab0d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.076185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.076870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.077191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.077522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] PATH_INFO: `/v3/projects/428b92251831427ba45c6a0d02d29c40/users/205501be938a451092f8c37b766b7f10/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.095320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428b92251831427ba45c6a0d02d29c40, user_id=205501be938a451092f8c37b766b7f10, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.098235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.113865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1129/2231] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:55 2022] PUT /identity/v3/projects/428b92251831427ba45c6a0d02d29c40/users/205501be938a451092f8c37b766b7f10/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:55.124490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.125101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.125101 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.157609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] MFA Rules not processed for user `205501be938a451092f8c37b766b7f10`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:55.158820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.177793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:55.229990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1103/2232] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:55.234623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.303344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04435-642a-4693-9321-e7aee0851d40 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.308053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.308665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.308945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.309245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.309881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.311768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.339245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1130/2233] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:55.344002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.355370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ad5a1-d581-4b5e-a09b-f73f6572b644 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.418892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.420126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.420417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.420700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.421420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.423341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.446157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1104/2234] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:55.450555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.469421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3fb5-3e2f-4748-94e5-f185f8397cd5 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.474461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.475687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.476047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.476321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.477191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.478841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.490492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1131/2235] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.494783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.505393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9f78-80af-4ec1-ab4e-83b0af625a44 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.510000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.510648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.510933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.511214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] PATH_INFO: `/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299/users/8467d4ca3759417f8aa5c5dc771951cd/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.526053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b2d459aa64497e8ba4aff2b4fd6299, user_id=8467d4ca3759417f8aa5c5dc771951cd, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.528841 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.542813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1105/2236] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:55 2022] PUT /identity/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299/users/8467d4ca3759417f8aa5c5dc771951cd/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:55.547495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.557516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c352d0-7019-4b3c-aa4f-04b66ff39d20 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.577928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:55.618083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.618678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.618965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.619240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.619874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.622083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af0fc0-0224-4d6a-bf28-97097edd3acc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.631977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1132/2237] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:55.636084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.646782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da3669-65cf-4c07-9b89-d1c0284f69b7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.651558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.652213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.652494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.652767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] PATH_INFO: `/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299/users/8467d4ca3759417f8aa5c5dc771951cd/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.654701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b2d459aa64497e8ba4aff2b4fd6299, user_id=8467d4ca3759417f8aa5c5dc771951cd, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.657385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.667535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1106/2238] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:55 2022] PUT /identity/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299/users/8467d4ca3759417f8aa5c5dc771951cd/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:55.679455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.679749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.680029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.708575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] MFA Rules not processed for user `8467d4ca3759417f8aa5c5dc771951cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:55.709844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.782049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1133/2239] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:55.793519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.795230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.854342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910c03f-9703-4e40-8ef6-9a0d6efa788f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.857980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.858590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.858869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.859142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.867929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.869448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.879397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1107/2240] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:55.964207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.965724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.977845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d8922-fe55-45fa-a454-d9c1df60dabb None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.981624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.982221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.982505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.982780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.041370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.042816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7bee7-851c-4b3a-9975-1530c9bf47b3 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.051644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1134/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:02.878008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.891557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:02.940109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea0b7-796f-4551-b795-e00a049e2171 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.944858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.945515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.945792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.946064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] PATH_INFO: `/v3/users/205501be938a451092f8c37b766b7f10` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.947161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205501be938a451092f8c37b766b7f10)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.949615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.994844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205501be938a451092f8c37b766b7f10'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.003636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205501be938a451092f8c37b766b7f10'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.015288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205501be938a451092f8c37b766b7f10'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.022252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205501be938a451092f8c37b766b7f10'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.022889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205501be938a451092f8c37b766b7f10'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.028634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1108/2242] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:02 2022] DELETE /identity/v3/users/205501be938a451092f8c37b766b7f10 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.033102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.033557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.033650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.052690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:03.053925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.095143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.113870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1135/2243] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:03.123492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.124873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.181823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6008a-9c4d-454a-af73-7af0d9d62ec7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.185477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.186082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.186364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.186645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.195670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.197102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.207049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1109/2244] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:03.470633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.472231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.483978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a4ebe-f160-4706-9871-b357a40f4b89 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.490120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.490915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.491168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.491400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] PATH_INFO: `/v3/projects/428b92251831427ba45c6a0d02d29c40` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.492535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428b92251831427ba45c6a0d02d29c40)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.497828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.544362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428b92251831427ba45c6a0d02d29c40', 'request_id': 'req-062a490f-004b-40d0-a633-f4745a28a7f2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.549772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428b92251831427ba45c6a0d02d29c40', 'request_id': 'req-062a490f-004b-40d0-a633-f4745a28a7f2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.555903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428b92251831427ba45c6a0d02d29c40', 'request_id': 'req-062a490f-004b-40d0-a633-f4745a28a7f2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.568869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] The token cache is being invalidate because project 428b92251831427ba45c6a0d02d29c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:03.569191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.569826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.571504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1136/2245] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:03 2022] DELETE /identity/v3/projects/428b92251831427ba45c6a0d02d29c40 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.575850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.585512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d849eb-0e58-4ddc-9387-03abe719cdad service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.616717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.643560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.644197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.644483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.644760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] PATH_INFO: `/v3/users/8467d4ca3759417f8aa5c5dc771951cd` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.645913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8467d4ca3759417f8aa5c5dc771951cd)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.647834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.692862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8467d4ca3759417f8aa5c5dc771951cd'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.700858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8467d4ca3759417f8aa5c5dc771951cd'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.714282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8467d4ca3759417f8aa5c5dc771951cd'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.721670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8467d4ca3759417f8aa5c5dc771951cd'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.722400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1cb8b6b2-1004-4719-96b4-b6afa111b6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8467d4ca3759417f8aa5c5dc771951cd'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.728442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1110/2246] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:03 2022] DELETE /identity/v3/users/8467d4ca3759417f8aa5c5dc771951cd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.085969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.087511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.103685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a490f-004b-40d0-a633-f4745a28a7f2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.175071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.175880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.176259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.176761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] PATH_INFO: `/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.178613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b2d459aa64497e8ba4aff2b4fd6299)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.181072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.243028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b2d459aa64497e8ba4aff2b4fd6299', 'request_id': 'req-dea6464a-3b34-4542-9705-58b04d23207f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.250607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b2d459aa64497e8ba4aff2b4fd6299', 'request_id': 'req-dea6464a-3b34-4542-9705-58b04d23207f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.258568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b2d459aa64497e8ba4aff2b4fd6299', 'request_id': 'req-dea6464a-3b34-4542-9705-58b04d23207f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.268284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:04.273871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] The token cache is being invalidate because project 14b2d459aa64497e8ba4aff2b4fd6299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:04.274391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.275793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dea6464a-3b34-4542-9705-58b04d23207f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.281568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1137/2247] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:04 2022] DELETE /identity/v3/projects/14b2d459aa64497e8ba4aff2b4fd6299 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.283110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.283578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.283682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.303687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.304932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.366257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1111/2248] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:04.378210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.378495 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.378765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.398414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.399604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.407924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1138/2249] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:04.412229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.422726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7012a1-8c16-41eb-b414-1760f01637c2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.427282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.427883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.428167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.428445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.429135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:04.429445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.431025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.442707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1112/2250] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:04.446767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.456223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe97d4-5061-4449-914e-97141165a4e1 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.460750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.461360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.461641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.461911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.462521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.464398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.490771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1139/2251] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:04.494896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.504852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f041de-5580-4d5a-9069-2b0be1457954 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.544822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:04.562306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.562915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.563199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.563481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.564108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.566106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.586596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1113/2252] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:04.591239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.601383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd39cb9-790e-4b21-8162-335f13fbeaab admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.605905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.606829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.607118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.607387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.608021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.609664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.619498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1140/2253] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:04.623512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.633638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe3252-4215-4659-a372-1ec05bd851e2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.638783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.639383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.639667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.639943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.640578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.642197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d560a-f50d-4e07-9bdd-eca7a25e78c2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.653277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1114/2254] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:04.657213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.667263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661357af-3326-4411-9b99-fc5c1a75f408 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.672947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.673696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.673977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.674253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] PATH_INFO: `/v3/projects/b7ba055e39ab422d8ffc478392fc751f/users/b32defb85df5495ebe6c3d550e38a0a2/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.688792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ba055e39ab422d8ffc478392fc751f, user_id=b32defb85df5495ebe6c3d550e38a0a2, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.691550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.705845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1141/2255] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:04 2022] PUT /identity/v3/projects/b7ba055e39ab422d8ffc478392fc751f/users/b32defb85df5495ebe6c3d550e38a0a2/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.717599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.717890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.718194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.745227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] MFA Rules not processed for user `b32defb85df5495ebe6c3d550e38a0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.746436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.809949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1115/2256] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:04.814582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.875319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c5723-1126-4583-88b3-14a4ea14ff31 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.879916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.880514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.880795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.881106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.881725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.883628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.913419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1142/2257] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:04.914390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.925257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aca77-9fd4-44fe-9682-4112097001f1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.982500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.983119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.983408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.983669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.984305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.986233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.009082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1116/2258] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:05.013232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.023928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928ea8-69ac-4bcb-9fce-b6c8ac116b6c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.031228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.031839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.032276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.032554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.033301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.034951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.045813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1143/2259] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:05.050282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.061615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213c27-cb7c-411c-b2fb-425546a1ea0c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.066342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.067026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.067562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.067836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] PATH_INFO: `/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6/users/07fa4cc2e7934dbeae7167790b4b2455/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.083916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6bd6cd10dd40a19b101077e9aeceb6, user_id=07fa4cc2e7934dbeae7167790b4b2455, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.086614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.099542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1117/2260] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:05 2022] PUT /identity/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6/users/07fa4cc2e7934dbeae7167790b4b2455/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.103610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.113614 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d324cb-b939-4501-b1c6-9df52192509d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.169276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.169805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.170087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.170366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.171030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.172671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5228be-ed94-49a6-a788-d47f3e745e73 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.182574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1144/2261] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:05.187212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.197691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23500d-72a6-4de8-ab67-31636d4a21b9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.202927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.203576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.203869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.204136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] PATH_INFO: `/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6/users/07fa4cc2e7934dbeae7167790b4b2455/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.206241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6bd6cd10dd40a19b101077e9aeceb6, user_id=07fa4cc2e7934dbeae7167790b4b2455, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.208891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.219105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1118/2262] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:05 2022] PUT /identity/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6/users/07fa4cc2e7934dbeae7167790b4b2455/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.286146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.286317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.286683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.295635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.317546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] MFA Rules not processed for user `07fa4cc2e7934dbeae7167790b4b2455`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:05.318753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.386450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1145/2263] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:05.399269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.401206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.406536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.463644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9ef5-2c4b-4d8e-92d8-c2c73e4ab415 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.467272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.467878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.468155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.468426 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.477103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.478509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.488482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1119/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:05.560446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.570244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc532d-6a9f-471b-91ab-3d84b01cddf0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.573881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.574477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.574756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.575027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.631099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.632538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a57189-e5d7-4a84-a5cb-35e16b154b88 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.641322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1146/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:05.902741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.963527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f0935-879d-473a-a885-24edc149dc14 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.968644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.969299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.969585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.969865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] PATH_INFO: `/v3/users/b32defb85df5495ebe6c3d550e38a0a2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.971011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32defb85df5495ebe6c3d550e38a0a2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.972925 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.017630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32defb85df5495ebe6c3d550e38a0a2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.025108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32defb85df5495ebe6c3d550e38a0a2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.038102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32defb85df5495ebe6c3d550e38a0a2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.044815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32defb85df5495ebe6c3d550e38a0a2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.045509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32defb85df5495ebe6c3d550e38a0a2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.051245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1120/2266] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/users/b32defb85df5495ebe6c3d550e38a0a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:06.055643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.055936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.056218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.079030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:06.080226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.140351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1147/2267] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:06.150348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.151805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.180213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.215106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd3d5c-b0c3-4a94-a69a-8f9fa2b3c687 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.218759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.219337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.219621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.220010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.229524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.231526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.241130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1121/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:06.523070 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.525150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.540503 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c14eb-b631-4f9a-80c0-269d01a380fe None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.545301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.545914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.546321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.546454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] PATH_INFO: `/v3/projects/b7ba055e39ab422d8ffc478392fc751f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.547813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ba055e39ab422d8ffc478392fc751f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.550477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.579967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.605084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ba055e39ab422d8ffc478392fc751f', 'request_id': 'req-e95061cc-98ff-45d8-9640-cff6359edc5d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.611162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ba055e39ab422d8ffc478392fc751f', 'request_id': 'req-e95061cc-98ff-45d8-9640-cff6359edc5d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.617319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ba055e39ab422d8ffc478392fc751f', 'request_id': 'req-e95061cc-98ff-45d8-9640-cff6359edc5d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.629395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] The token cache is being invalidate because project b7ba055e39ab422d8ffc478392fc751f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:06.629685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.630301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.631779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1148/2269] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:06 2022] DELETE /identity/v3/projects/b7ba055e39ab422d8ffc478392fc751f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:06.635911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.646560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f639b-d8c1-42a2-a0a9-4cac435f8907 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.703670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.704299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.704580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.704859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] PATH_INFO: `/v3/users/07fa4cc2e7934dbeae7167790b4b2455` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.706008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fa4cc2e7934dbeae7167790b4b2455)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.707901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.753213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fa4cc2e7934dbeae7167790b4b2455'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.760206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fa4cc2e7934dbeae7167790b4b2455'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.772855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fa4cc2e7934dbeae7167790b4b2455'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.780410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fa4cc2e7934dbeae7167790b4b2455'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.781165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d7288070-4250-478e-8589-b3ad8280e854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fa4cc2e7934dbeae7167790b4b2455'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.786986 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1122/2270] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:06 2022] DELETE /identity/v3/users/07fa4cc2e7934dbeae7167790b4b2455 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:07.052083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.053701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.068602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95061cc-98ff-45d8-9640-cff6359edc5d admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.132258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.132894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.133285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.133548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] PATH_INFO: `/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.134665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6bd6cd10dd40a19b101077e9aeceb6)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.137171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.181920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6bd6cd10dd40a19b101077e9aeceb6', 'request_id': 'req-59f33a12-0e93-4200-b61f-d46ca9cb83bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.188062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6bd6cd10dd40a19b101077e9aeceb6', 'request_id': 'req-59f33a12-0e93-4200-b61f-d46ca9cb83bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.194504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6bd6cd10dd40a19b101077e9aeceb6', 'request_id': 'req-59f33a12-0e93-4200-b61f-d46ca9cb83bd'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.207767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] The token cache is being invalidate because project ea6bd6cd10dd40a19b101077e9aeceb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:07.208060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.208689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-59f33a12-0e93-4200-b61f-d46ca9cb83bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:07.210230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1149/2271] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:07 2022] DELETE /identity/v3/projects/ea6bd6cd10dd40a19b101077e9aeceb6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:07.215161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.215450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.215726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.235859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.237105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.282153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.308744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1123/2272] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:07.310540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.310833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.311108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.331359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.332565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.340383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1150/2273] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:07.344546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.354391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60293a-fa03-451f-9ac2-37cbfd1422b0 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.359060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.359693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.359968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.360237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.360876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:07.361209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.362788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.375255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1124/2274] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:07.379813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.389922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef977057-6665-46c7-90f6-24cd19735435 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.395364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.396076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.396437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.396777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.397737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.399754 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.433983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1151/2275] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:07.438147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.448254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9ae6c-f3ea-45ce-ad1d-c20c7793fef0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.503966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.504571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.504847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.505147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.505766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.507694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.528633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1125/2276] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:07.532938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.544027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7afaa-0b5b-4b09-830f-0efa9769540e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.548636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.549274 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.549559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.549838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.550476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.552101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.562604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1152/2277] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:07.566701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.577116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0e402-629d-4aea-98b6-9a250a81882a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.581827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.582428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.582707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.582998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.583609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.585209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b225bb-e42e-4884-9448-283e2e0b0d04 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.595969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1126/2278] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:07.600078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.605715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.612053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b0393-dee1-458c-a119-2eb8269f0825 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.617349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.617997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.618282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.618560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] PATH_INFO: `/v3/projects/036e74c1200f42aa97987e99ba3dfe85/users/fe99c2600f9e4b24bef88d8d76104013/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.632906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036e74c1200f42aa97987e99ba3dfe85, user_id=fe99c2600f9e4b24bef88d8d76104013, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.635603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.648157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1153/2279] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:07 2022] PUT /identity/v3/projects/036e74c1200f42aa97987e99ba3dfe85/users/fe99c2600f9e4b24bef88d8d76104013/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:07.653797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414fd-ec2e-4033-9282-4a8b87ac8cf5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.654422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414fd-ec2e-4033-9282-4a8b87ac8cf5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.654422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414fd-ec2e-4033-9282-4a8b87ac8cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.672938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-92f414fd-ec2e-4033-9282-4a8b87ac8cf5 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.674150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-92f414fd-ec2e-4033-9282-4a8b87ac8cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.731502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1127/2280] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:07.741088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.742869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.800702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c2dc3-a5cc-4d97-a7dd-f5cc62a11422 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.804330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.804933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.805235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.805507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.814740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.816108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.826044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1154/2281] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:08.429345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4176da-fdc4-48b8-848e-1e731fca2fac None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.429638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4176da-fdc4-48b8-848e-1e731fca2fac None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.429975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4176da-fdc4-48b8-848e-1e731fca2fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.465170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2e4176da-fdc4-48b8-848e-1e731fca2fac None None] MFA Rules not processed for user `fe99c2600f9e4b24bef88d8d76104013`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.466371 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2e4176da-fdc4-48b8-848e-1e731fca2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.515462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:08.547557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1128/2282] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:08.559331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.563445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.641309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f0a53-3940-4c11-835b-9d20ea343a56 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.646100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.646710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.646992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.647261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.656685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.658145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.666901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1155/2283] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:15.275478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10d07-032a-4d18-9a37-64774a6b94bc None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.276065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10d07-032a-4d18-9a37-64774a6b94bc None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.276172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c10d07-032a-4d18-9a37-64774a6b94bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.304138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-39c10d07-032a-4d18-9a37-64774a6b94bc None None] MFA Rules not processed for user `fe99c2600f9e4b24bef88d8d76104013`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:15.305384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-39c10d07-032a-4d18-9a37-64774a6b94bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:15.313024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1129/2284] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:18.663010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:19.488375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.489859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.503978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44226-1c9f-4643-9438-10dbad2a918e service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.508412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.509066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.509352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.509624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] PATH_INFO: `/v3/users/29c72f42695a4f5e868e1ffe57332fcf` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.510727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c72f42695a4f5e868e1ffe57332fcf)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.512679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.561840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c72f42695a4f5e868e1ffe57332fcf'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.562577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c72f42695a4f5e868e1ffe57332fcf'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.576343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c72f42695a4f5e868e1ffe57332fcf'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.583516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c72f42695a4f5e868e1ffe57332fcf'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.590373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c72f42695a4f5e868e1ffe57332fcf'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.597759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1156/2285] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:19 2022] DELETE /identity/v3/users/29c72f42695a4f5e868e1ffe57332fcf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:19.608232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.608232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.608232 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.623282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:19.624506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.687675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1130/2286] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:19.697865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.699402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.762898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3e957-95f0-48fc-9b54-e9f679099ada admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.766960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.768159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.768159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.768354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.778545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.779972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772b777-fdc1-4d76-a917-5ef79aa52a18 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.791254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1157/2287] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:20.170654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.172132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.186174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7e4b-161d-492a-9996-cdb7ca2bc12d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.190831 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.192389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.192662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.193187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] PATH_INFO: `/v3/projects/c761c2cc031f4bd6b6d652e88cff6578` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.194996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c761c2cc031f4bd6b6d652e88cff6578)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.196860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.243414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c761c2cc031f4bd6b6d652e88cff6578', 'request_id': 'req-9df5065b-cbfe-4ae2-9e22-92883e02f128'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.250615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c761c2cc031f4bd6b6d652e88cff6578', 'request_id': 'req-9df5065b-cbfe-4ae2-9e22-92883e02f128'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.257572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c761c2cc031f4bd6b6d652e88cff6578', 'request_id': 'req-9df5065b-cbfe-4ae2-9e22-92883e02f128'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.268927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] The token cache is being invalidate because project c761c2cc031f4bd6b6d652e88cff6578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:20.269248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.269905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9df5065b-cbfe-4ae2-9e22-92883e02f128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.271629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1131/2288] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:20 2022] DELETE /identity/v3/projects/c761c2cc031f4bd6b6d652e88cff6578 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:20.276832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.277165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.277448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.298187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.299442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.360756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1158/2289] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:20.368533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.368821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.369130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.389037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.390217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.397946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1132/2290] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:20.402275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.412049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdd86f-2654-4b7c-9e73-afb948818294 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.416747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.417872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.418151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.418422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.419060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:20.419361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.420979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.433201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1159/2291] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:20.437143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.446468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe6e71-74db-4a32-b443-45f0ab6e8a51 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.451045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.451633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.451911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.452182 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.452799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.454711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.484012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1133/2292] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:20.487308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.492442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:20.499928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14a7f1-7192-4054-b5e0-d334ea969d94 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.557854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.558183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.558476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.558743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.559366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.561400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.586038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1160/2293] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:20.587262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.598130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ae0d-c48b-4110-84bb-17f756214d77 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.602735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.603332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.603611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.603881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.604566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.606196 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.616563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1134/2294] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:20.621115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.631249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a44732-3723-4d3d-bf48-c44172b98c8b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.635892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.636517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.636861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.637208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.637843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.639453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b5cac-1894-4b29-9a43-c3d7948fe10a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.649924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1161/2295] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:20.654410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.664270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0e517-c8a0-43ec-9c49-85dc8dc3f9a5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.669918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.670575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.670852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.671262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] PATH_INFO: `/v3/projects/9df6e3fe84eb42b4a1a70e76b5de9fa2/users/5669f5e5dcd74e01b4fb7c1bd512bd82/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.686555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df6e3fe84eb42b4a1a70e76b5de9fa2, user_id=5669f5e5dcd74e01b4fb7c1bd512bd82, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.689268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.692513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:20.707520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1135/2296] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:20 2022] PUT /identity/v3/projects/9df6e3fe84eb42b4a1a70e76b5de9fa2/users/5669f5e5dcd74e01b4fb7c1bd512bd82/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:20.709171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.709622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.709789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.733067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.734277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.819367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1162/2297] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:20.844126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.846838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.928492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3232d53-f03a-4689-9021-0bce811d51c4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.933342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.934160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.934605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.934937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.945319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.946892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5831a6-4116-4451-9e2d-58cd41ced529 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.957280 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1136/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:24.405678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.407124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.420637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d50cf-06a4-486a-88c3-dfb996c3e70b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.428609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.429566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.430092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.430553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] PATH_INFO: `/v3/users/3356913cfe8e40a5b4bb24a720b40771` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.431973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3356913cfe8e40a5b4bb24a720b40771)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.434720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.517997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3356913cfe8e40a5b4bb24a720b40771'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.519055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3356913cfe8e40a5b4bb24a720b40771'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.551227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3356913cfe8e40a5b4bb24a720b40771'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.571368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3356913cfe8e40a5b4bb24a720b40771'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.579214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:24.599588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3356913cfe8e40a5b4bb24a720b40771'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.617169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1163/2299] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:24 2022] DELETE /identity/v3/users/3356913cfe8e40a5b4bb24a720b40771 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:24.622263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbdcd-e751-4b47-80df-eee780414b46 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.622799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbdcd-e751-4b47-80df-eee780414b46 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.622799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffbdcd-e751-4b47-80df-eee780414b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.658377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f5ffbdcd-e751-4b47-80df-eee780414b46 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:24.661755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f5ffbdcd-e751-4b47-80df-eee780414b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.771011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1137/2300] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:24.785035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.786275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.879656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2b50-48f2-4861-ae96-36138233585b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.886149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.887523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.888184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.888610 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.907002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.909294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.927452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1164/2301] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:25.345315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017cd8-6be8-4708-87f5-c2a48edbc3ef None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.345608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017cd8-6be8-4708-87f5-c2a48edbc3ef None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.345890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017cd8-6be8-4708-87f5-c2a48edbc3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.379962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0b017cd8-6be8-4708-87f5-c2a48edbc3ef None None] MFA Rules not processed for user `79acdbc2c11c4459928eab5b83a6755d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.381292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0b017cd8-6be8-4708-87f5-c2a48edbc3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.461121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1138/2302] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:25.580531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.582532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.597603 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a0b17-fbc5-43d9-8909-3e1bc035db6f service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.602804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.605391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.605869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.606856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] PATH_INFO: `/v3/projects/79311488c215429192c50198b51eeb10` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.608577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79311488c215429192c50198b51eeb10)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.610666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.689216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79311488c215429192c50198b51eeb10', 'request_id': 'req-61e1863c-1f8b-4756-a107-f41385e44315'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:25.698558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79311488c215429192c50198b51eeb10', 'request_id': 'req-61e1863c-1f8b-4756-a107-f41385e44315'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:25.711637 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79311488c215429192c50198b51eeb10', 'request_id': 'req-61e1863c-1f8b-4756-a107-f41385e44315'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:25.728046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] The token cache is being invalidate because project 79311488c215429192c50198b51eeb10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:25.728343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:25.729840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:25.734745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1165/2303] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:25 2022] DELETE /identity/v3/projects/79311488c215429192c50198b51eeb10 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:26.353358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.353866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.353954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.399885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] MFA Rules not processed for user `5669f5e5dcd74e01b4fb7c1bd512bd82`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:26.402441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.449694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:26.486595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1139/2304] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:26.490769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.492285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:26.622464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1863c-1f8b-4756-a107-f41385e44315 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.628549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.629279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.629563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.629835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.630477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.632432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.679590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1166/2305] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:26.680213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.714230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ad87a-10e0-4a89-94dd-8b1bb1ce33db None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.813415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.814618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.814909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.815197 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.815853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.818290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.855178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1140/2306] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:26 2022] POST /identity/v3/users => generated 295 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:26.859022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.869012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:26.876252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3b6b-0f77-4c1e-9450-68ca18feb65e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.897795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.901870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.902349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.902818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.905065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.910705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:26.934646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1167/2307] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:26.938946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:26.956287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0215dce-b1bc-48de-9d48-adcab68ea4b5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:26.961473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:26.962633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:26.963113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:26.963627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] PATH_INFO: `/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0/users/fd3a2c62a61a4652ad3f70e70788470f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:26.986330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb198f9758f4ad5af15ced1455ec6e0, user_id=fd3a2c62a61a4652ad3f70e70788470f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:26.989798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.015092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1141/2308] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:26 2022] PUT /identity/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0/users/fd3a2c62a61a4652ad3f70e70788470f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:27.018793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.030046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a0eea-87fa-4b9b-81fa-5fcab3586fde admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.182761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.184582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:27.193482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.194127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.194390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.194825 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.195427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.197322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.209545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1168/2309] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:27.213636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:27.224913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68099e9-2d3b-40c4-a7ad-a7fa0361f1da admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.230700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.231841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.232223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.232596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] PATH_INFO: `/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0/users/fd3a2c62a61a4652ad3f70e70788470f/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.234952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb198f9758f4ad5af15ced1455ec6e0, user_id=fd3a2c62a61a4652ad3f70e70788470f, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.238088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72730eb-76d9-44e5-8114-f955de857e28 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.256413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1169/2310] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:27 2022] PUT /identity/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0/users/fd3a2c62a61a4652ad3f70e70788470f/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:27.299465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2cbd2-9f66-475e-a6ce-95649e85e15a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:27.350801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:27.374282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:27.375391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:27.375391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:27.375640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:27.377851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:27.389505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:27.391121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:27.404379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1142/2311] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:32.806024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba2b-6150-43dc-a110-378d6db0a4d9 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.806541 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba2b-6150-43dc-a110-378d6db0a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.806715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba2b-6150-43dc-a110-378d6db0a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.838873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-801fba2b-6150-43dc-a110-378d6db0a4d9 None None] MFA Rules not processed for user `fd3a2c62a61a4652ad3f70e70788470f`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:32.840400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-801fba2b-6150-43dc-a110-378d6db0a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:32.919075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1170/2312] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:32.935215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:32.937402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.015927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a64bf-b1b0-48bc-8689-14c77ea8eca5 service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.022064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.022700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.022982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.024248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.088910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.090353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ac46-4057-42ff-990f-7b84365e9506 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.100154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1143/2313] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:41.505333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e14cc-49cf-48d2-b9c0-6842f5f6e5d9 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:41.505815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e14cc-49cf-48d2-b9c0-6842f5f6e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:41.505911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e14cc-49cf-48d2-b9c0-6842f5f6e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:41.525840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-483e14cc-49cf-48d2-b9c0-6842f5f6e5d9 None None] MFA Rules not processed for user `fe99c2600f9e4b24bef88d8d76104013`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:41.527063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-483e14cc-49cf-48d2-b9c0-6842f5f6e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:41.583496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1171/2314] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:49:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:43.095745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=69 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:51.576656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.508307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.508943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.509083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.528212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] MFA Rules not processed for user `5669f5e5dcd74e01b4fb7c1bd512bd82`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:01.529431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.536138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1144/2315] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:50:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:11.523003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:13.883987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.884534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.884690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.904243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:13.905556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:13.964797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1172/2316] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:14.008523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.010283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:14.084780 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad499e-6c98-46ac-8443-609b2468d9e5 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.091015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.091136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.091326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.091605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.100060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.102286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98630-4743-4fc3-95f7-6c3c45630ae4 service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.110502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1145/2317] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:23.955733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=25 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:24.107318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=32 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:31.334873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:31.336768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:31.414038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d540284-ccdf-47a7-a11c-bd8df7b2beb5 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:31.417963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:31.418577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:31.418861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:31.419132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:31.429359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:31.430978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:31.441512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1173/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:34.044982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:34.045538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:34.045645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:34.067450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:34.068714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:34.081160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1146/2319] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 09:50:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:34.100380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:34.113349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a7a32-cb00-471d-81a4-1754e401d920 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:34.120073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:34.120903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:34.121222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:34.121498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:34.130708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:34.132935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:34.141719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1174/2320] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:36.228148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:36.229743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:36.297006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4700c-7aaf-47e3-9a9f-7b3a302ac1b7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:36.301839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:36.302500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:36.302785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:36.303061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] PATH_INFO: `/v3/users/fe99c2600f9e4b24bef88d8d76104013` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:36.305020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe99c2600f9e4b24bef88d8d76104013)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:36.307080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:36.359007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe99c2600f9e4b24bef88d8d76104013'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.372221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe99c2600f9e4b24bef88d8d76104013'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.385055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe99c2600f9e4b24bef88d8d76104013'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.394695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe99c2600f9e4b24bef88d8d76104013'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.395248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-4d15bfc5-91af-4f54-97e3-df0061050cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe99c2600f9e4b24bef88d8d76104013'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.401746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1147/2321] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:36 2022] DELETE /identity/v3/users/fe99c2600f9e4b24bef88d8d76104013 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:36.579830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:36.593888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d1727-134a-4971-9a9e-73529ee0a646 service swift] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:36.652633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:36.653310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:36.653591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:36.653978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] PATH_INFO: `/v3/projects/036e74c1200f42aa97987e99ba3dfe85` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:36.655287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036e74c1200f42aa97987e99ba3dfe85)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:36.658450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:36.709277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036e74c1200f42aa97987e99ba3dfe85', 'request_id': 'req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.714957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036e74c1200f42aa97987e99ba3dfe85', 'request_id': 'req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.721456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036e74c1200f42aa97987e99ba3dfe85', 'request_id': 'req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.736698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] The token cache is being invalidate because project 036e74c1200f42aa97987e99ba3dfe85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:36.737635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.738283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39cf3742-d256-40dc-bf77-84bc1fcd7dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:36.740037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1175/2322] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:36 2022] DELETE /identity/v3/projects/036e74c1200f42aa97987e99ba3dfe85 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:38.822228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:38.822676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:38.822806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:38.841767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] MFA Rules not processed for user `0fca332203b9473fa8b07286bc44300f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:38.842945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:38.895909 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1148/2323] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:50:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:46.733586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:48.889237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:49.277479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bc843-9ce8-4e93-adbd-f5ca10213b28 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.277976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bc843-9ce8-4e93-adbd-f5ca10213b28 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.278093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bc843-9ce8-4e93-adbd-f5ca10213b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.299351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-192bc843-9ce8-4e93-adbd-f5ca10213b28 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:49.300553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-192bc843-9ce8-4e93-adbd-f5ca10213b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.355106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1176/2324] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:49.367624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.369179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.438350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7ea0-14d5-4e32-bede-0681f534a028 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.443385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.443990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.444269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.444539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.452436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.454044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed600c9-a012-4daf-8507-13a349dcd37d service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.462396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1149/2325] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:50:59.348026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=25 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:59.459215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=32 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:06.359228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.359808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.359808 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.361142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1177/2326] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:51:06.366916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b697-09cc-41d1-89b0-6099213b5313 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.367387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b697-09cc-41d1-89b0-6099213b5313 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.367640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b697-09cc-41d1-89b0-6099213b5313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.390281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5792b697-09cc-41d1-89b0-6099213b5313 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:06.391438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5792b697-09cc-41d1-89b0-6099213b5313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.474157 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1150/2327] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:51:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:06.486933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:06.488445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.553782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb2e4-6c04-484c-b04a-bb339cd8025a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:06.557371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:06.557980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.558261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.558536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.569233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:06.570661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18281175-1876-4ce7-9cfd-213210b3ae90 service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:06.581135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1178/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:06.649229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.649444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.649722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.651050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1151/2329] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:51:06.655764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1be01-81d3-4c56-832c-a9f481ca4cca None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.656053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1be01-81d3-4c56-832c-a9f481ca4cca None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.656323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1be01-81d3-4c56-832c-a9f481ca4cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.675823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1da1be01-81d3-4c56-832c-a9f481ca4cca None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:06.677575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1da1be01-81d3-4c56-832c-a9f481ca4cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.727020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1179/2330] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:51:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:06.738876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:06.740568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.753781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e6c57-34dc-4483-a84c-a6c3d7a7dd95 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:06.757422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:06.758013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.758295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.758570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.765532 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:06.766938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8944aa4-36c9-4b6b-8325-7a997bb6639e service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:06.775111 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1152/2331] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:51:06.862050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.862339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.862616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.864008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1180/2332] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:51:06.868976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434222a-4d38-42c9-81c6-5c387dfd33e7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.869290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434222a-4d38-42c9-81c6-5c387dfd33e7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.869576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434222a-4d38-42c9-81c6-5c387dfd33e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.888760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6434222a-4d38-42c9-81c6-5c387dfd33e7 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:06.889978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6434222a-4d38-42c9-81c6-5c387dfd33e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.899634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1153/2333] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:51:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:06.909918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:06.912188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.925226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43c169-af17-4abb-9efe-e7ed4dcdb719 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:06.928701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:06.929340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.929601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.929870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.939586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:06.941349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:06.951573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1181/2334] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:10.294334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.294789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.294880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.313592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:10.315038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:10.322585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1154/2335] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:51:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:10.334872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.346032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da48ab4-e80a-46fd-b340-8c09286c66f2 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.350989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.351581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.351859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.352132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.359964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.361582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.369423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1182/2336] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:51:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:51:20.308368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=43 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:20.366344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=61 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:22.917333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:22.928845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2754c5-432d-499b-8a9f-4d1ef21e5b2c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:22.933381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:22.934005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:22.934287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:22.934565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:22.936347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:22.951004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:22.952491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:22.964404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1155/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:26.505010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:26.514266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc973ed-cf0c-4d0c-b958-17a483cfc23c service swift] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:26.517831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:26.518424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:26.518703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:26.518973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:26.520712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:26.533621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:26.535048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:26.545604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1183/2338] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:31.451626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:31.453225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:31.467239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf353-bbd2-406a-9b38-7a72430c6202 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:31.473384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:31.473974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:31.474252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:31.474521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:31.476160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:31.531844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:31.533483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2e1a1-e685-4f1b-b6d8-b6d17921e7fe service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:31.543053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1156/2339] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:36.540997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:39.764922 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:39.774595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66075601-655f-47f7-8134-19e80aed1a55 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:39.778172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:39.778778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:39.779061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:39.779333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:39.781121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:39.790760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:39.792188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:39.800996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1184/2340] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:41.538916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=34 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:41.784703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:41.785095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:41.785445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:41.804616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:41.805835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:41.812864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1157/2341] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 09:51:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:41.837835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:41.851703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91c5e0-7fca-4f4d-88fe-1480f09eeeae service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:41.856929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:41.857570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:41.857857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:41.858132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:41.866967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:41.868552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c7c7e-4927-4000-b1ad-6b14d98e7589 service swift] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:41.876702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1185/2342] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:51:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:51:46.633955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:46.647094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50369082-c765-4ff1-b8b4-d3edda652834 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:46.652451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:46.653150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:46.653431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:46.653701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:46.655383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:46.714091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:46.715699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:46.725553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1158/2343] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:51.873344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=12 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:56.720650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=31 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:12.323316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:12.323843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:12.323843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:12.343063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] MFA Rules not processed for user `79acdbc2c11c4459928eab5b83a6755d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:12.344195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:12.351674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1186/2344] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:52:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:15.694413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:15.706364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09485b1-74e6-4e36-867b-21257863c114 service nova] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:15.712091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:15.712700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:15.713058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:15.713364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:15.715090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:15.772697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:15.774244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:15.783387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1159/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:18.587525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:18.605417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6761e1-c600-43ba-ac28-e130f67a5693 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:18.611951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:18.612567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:18.613051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:18.613504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:18.615438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:18.630415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:18.632182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:18.643605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1187/2346] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:25.779050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=28 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:28.638488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=9 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:45.848019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:45.859154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00522b49-77eb-4530-9b29-84240d603309 service nova] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:45.865268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:45.865873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:45.866159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:45.866435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:45.868167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:45.878524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:45.880091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff6828-0f74-4091-a02b-db97017e1867 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:45.889245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1160/2347] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:55.884921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:57.618240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:57.631700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93d954-bb89-4056-af50-cc30b3d3c5c8 service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:57.638833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:57.639455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:57.640393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:57.640766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:57.643133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:57.658627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:57.660804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13b03d-d31f-4ee3-98ea-f7c059735c05 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:57.677916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1188/2348] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:01.260793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:01.261400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:01.261400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:01.281956 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] MFA Rules not processed for user `0fca332203b9473fa8b07286bc44300f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:01.283171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:01.291441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1161/2349] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:01.365429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:01.365885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:01.365997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:01.387932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] MFA Rules not processed for user `468522570ba7419c87da6d05fc0d1ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:01.389167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:01.396543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1189/2350] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:01.627393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:01.636806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f0b9e-af4d-4228-8c01-71481633b24b None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:01.640901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:01.641529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:01.641812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:01.642081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:01.643826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:01.653431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:01.654838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:01.662977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1162/2351] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:01.707429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:01.716923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c4a49-2783-455f-8e58-f82715418b02 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:01.720521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:01.721150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:01.721433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:01.721708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:01.723425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:01.733942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:01.735378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:01.743459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1190/2352] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:01.990454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:01.992079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:02.107387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:02.109731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:02.143513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb02b4-7a31-44dc-9682-7c7a91b263f0 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:02.156993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:02.159218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:02.159907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:02.160443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:02.162842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:02.178838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:02.180732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:02.186904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f2fa-7aac-4c4d-a279-6a73b5d8dbb4 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:02.192282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:02.192891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:02.193201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:02.193480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:02.195851 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1163/2353] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:02.202290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:02.203871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c94a1-f5cc-4fc9-b67b-b960ff920caa service placement] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:02.213523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1191/2354] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:53:12.188929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=41 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:12.209344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=47 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:13.676406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:13.690697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc5ad-e120-4fbe-a727-af7b61dfb4c4 service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:13.696238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:13.697017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:13.697334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:13.697647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:13.699362 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:13.712658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:13.714298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:13.727636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1164/2355] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:23.723148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:34.955298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d958-f257-4781-9d9d-23a18d6192cb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:34.955783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d958-f257-4781-9d9d-23a18d6192cb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:34.955877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4d958-f257-4781-9d9d-23a18d6192cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:34.975544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3ad4d958-f257-4781-9d9d-23a18d6192cb None None] MFA Rules not processed for user `5669f5e5dcd74e01b4fb7c1bd512bd82`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:34.976740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3ad4d958-f257-4781-9d9d-23a18d6192cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:35.039553 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1192/2356] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:45.032764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=25 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:50.920128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:50.929610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028df4-af44-45e3-872b-232fd17daff0 service nova] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:50.933201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:50.934824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:50.934824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:50.935106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:50.940230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:51.020812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:51.022691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb1b5e-7e27-407f-bced-673beb3a097a service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:51.037141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1165/2357] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:55.131788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337886-4f68-4c2b-9502-46a28bdb980b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:55.132680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337886-4f68-4c2b-9502-46a28bdb980b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:55.133196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96337886-4f68-4c2b-9502-46a28bdb980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:55.157099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-96337886-4f68-4c2b-9502-46a28bdb980b None None] MFA Rules not processed for user `a0101b6f23d04016a89c986621ba5dde`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:55.159911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-96337886-4f68-4c2b-9502-46a28bdb980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:55.181285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1193/2358] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:55.743652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67214f-7437-42a6-96db-194ddb7ebcfb None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:55.743652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67214f-7437-42a6-96db-194ddb7ebcfb None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:55.743652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67214f-7437-42a6-96db-194ddb7ebcfb None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:55.744410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1166/2359] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 09:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:53:55.779897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:55.781039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:55.781440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:55.807137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] MFA Rules not processed for user `be582810785a49d48b0a06cd66b316a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:55.808407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:55.821494 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1194/2360] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 09:53:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:55.849412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67214f-7437-42a6-96db-194ddb7ebcfb None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:55.873600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67214f-7437-42a6-96db-194ddb7ebcfb None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:55.884934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:55.887514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:55.888790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:55.889654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:55.890143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:55.890826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:55.909439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0f2b0-9c0d-45c6-a8e2-5fcde43d0055 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:55.914763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:55.915653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:55.916246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:55.916696 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] PATH_INFO: `/v3/users/79acdbc2c11c4459928eab5b83a6755d` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:55.918095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:55.918365 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79acdbc2c11c4459928eab5b83a6755d)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:55.923781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:55.924392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:55.949955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1167/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:56.028331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79acdbc2c11c4459928eab5b83a6755d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.029102 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79acdbc2c11c4459928eab5b83a6755d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.046016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79acdbc2c11c4459928eab5b83a6755d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.056043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79acdbc2c11c4459928eab5b83a6755d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.069814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6f1f6c4b-bcd4-4a1a-aacf-c2034c540af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79acdbc2c11c4459928eab5b83a6755d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.083818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1195/2362] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:53:55 2022] DELETE /identity/v3/users/79acdbc2c11c4459928eab5b83a6755d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:56.295698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.310017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f575-4e5b-470c-a573-b0f3c94012c9 service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.386238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.386928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.387216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.387661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] PATH_INFO: `/v3/projects/089a845bce2542969b7e6f013b28d928` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.388858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] RBAC: Authorizing `identity:delete_project(project_id=089a845bce2542969b7e6f013b28d928)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.390802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.457681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089a845bce2542969b7e6f013b28d928', 'request_id': 'req-9908dd62-68af-4a6e-a45b-345ac3ee573c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.467538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089a845bce2542969b7e6f013b28d928', 'request_id': 'req-9908dd62-68af-4a6e-a45b-345ac3ee573c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.474563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089a845bce2542969b7e6f013b28d928', 'request_id': 'req-9908dd62-68af-4a6e-a45b-345ac3ee573c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.486980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] The token cache is being invalidate because project 089a845bce2542969b7e6f013b28d928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:53:56.487277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.488945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9908dd62-68af-4a6e-a45b-345ac3ee573c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:56.494818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1168/2363] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:53:56 2022] DELETE /identity/v3/projects/089a845bce2542969b7e6f013b28d928 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:56.496573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.497065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.497276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.517881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:56.519109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:56.585867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1196/2364] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:56.594527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.594790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.595072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.617496 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:56.618776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:56.630216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1169/2365] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:56.634702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.647911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fa6bd-3af1-4943-abc2-0bfe88580277 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.654063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.655268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.655650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.656020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.656799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:53:56.657248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.659139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.673612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1197/2366] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:53:56.678114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.684091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:56.689326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050ab0-240b-464e-8914-7710d56d5ceb None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.696357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.697135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.697678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.698120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.699621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.701754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.735967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1170/2367] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:53:56 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:53:56.737581 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.748005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee65c6-8b58-4dc8-9c01-7f4a48f7805c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.800820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:56.816983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.817620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.817906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.818181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.818817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.820993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.846612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1198/2368] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:53:56 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:53:56.850779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.861937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a4815-5cd4-465b-8468-2874e74911d6 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.866693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.867316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.867600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.867874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.868530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.870168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.881913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1171/2369] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:53:56.886109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.897782 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c220-77e5-4d46-ba60-717dcb7342e2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.902643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.903267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.903550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.903826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.904468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.906179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5805c8-4408-426a-966b-d372fbc0f379 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.917725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1199/2370] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:53:56.922172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:56.932884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7675540-2573-493c-bc3d-3d77bcf9ab1a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:56.938574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:56.939331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.939689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.940035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] PATH_INFO: `/v3/projects/67b53fced7324bfe996e98857a4d3074/users/a5d58cd8d7794e279c1b4952072f7c8a/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:56.957245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b53fced7324bfe996e98857a4d3074, user_id=a5d58cd8d7794e279c1b4952072f7c8a, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:56.959911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:56.974027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1172/2371] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:53:56 2022] PUT /identity/v3/projects/67b53fced7324bfe996e98857a4d3074/users/a5d58cd8d7794e279c1b4952072f7c8a/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:56.985199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:56.985566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:56.985903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:57.020569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] MFA Rules not processed for user `a5d58cd8d7794e279c1b4952072f7c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:57.022490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:57.129580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1200/2372] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:57.137607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:57.139462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:57.269149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38cda2-7c5c-49ba-8150-452e0c157757 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:57.275941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:57.276572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:57.276854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:57.277192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:57.288124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:57.290975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d47cf-d9a5-4f4d-9b30-957cdc486174 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:57.304501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1173/2373] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:57.551896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:57.641083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c32d-1caf-48ed-9ae7-cbb9e2f7d9e0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:57.646357 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:57.647254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:57.647611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:57.647959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] PATH_INFO: `/v3/users/a5d58cd8d7794e279c1b4952072f7c8a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:57.649302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d58cd8d7794e279c1b4952072f7c8a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:57.651462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:57.712367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d58cd8d7794e279c1b4952072f7c8a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:57.713111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d58cd8d7794e279c1b4952072f7c8a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:57.721418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:57.729249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d58cd8d7794e279c1b4952072f7c8a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:57.738027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d58cd8d7794e279c1b4952072f7c8a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:57.751157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d58cd8d7794e279c1b4952072f7c8a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:57.759108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1201/2374] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:53:57 2022] DELETE /identity/v3/users/a5d58cd8d7794e279c1b4952072f7c8a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:53:57.764317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:57.764613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:57.764893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:57.788136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:57.789596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:57.869733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1174/2375] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:57.883847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:57.888391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:57.977399 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e11a3-4858-47b7-8cde-b457e9b9ff6b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:57.983386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:57.984502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:57.984787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:57.985106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.000117 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:58.002123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77c2c0-5830-472e-b8ed-90c11773b5e3 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:58.016390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1202/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:58.650040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:58.651846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:58.676014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f894ec7-b595-4c6e-81f7-a95356c206c7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:58.697090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:58.698032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:58.698319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:58.698597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] PATH_INFO: `/v3/projects/67b53fced7324bfe996e98857a4d3074` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.699913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b53fced7324bfe996e98857a4d3074)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:58.702061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:58.741762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:58.762256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b53fced7324bfe996e98857a4d3074', 'request_id': 'req-f7935f01-aa4c-4517-aa14-acd0b19b5f65'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:58.770543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b53fced7324bfe996e98857a4d3074', 'request_id': 'req-f7935f01-aa4c-4517-aa14-acd0b19b5f65'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:58.777547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b53fced7324bfe996e98857a4d3074', 'request_id': 'req-f7935f01-aa4c-4517-aa14-acd0b19b5f65'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:58.787962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] The token cache is being invalidate because project 67b53fced7324bfe996e98857a4d3074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:53:58.788633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:58.789054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f7935f01-aa4c-4517-aa14-acd0b19b5f65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:53:58.790701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1175/2377] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:53:58 2022] DELETE /identity/v3/projects/67b53fced7324bfe996e98857a4d3074 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:58.795797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:58.796304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:58.796423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.818297 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:58.819569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:58.880214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1203/2378] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:58.888535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:58.889066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:58.889190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.916897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:58.918165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:58.926545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1176/2379] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:58.930961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:58.940868 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb81994-729d-4347-ba86-93a0749f96da None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:58.945427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:58.946139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:58.946347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:58.946657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.947340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:53:58.948407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:58.950788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:58.962787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1204/2380] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:53:58.967874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:58.977258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d97d59-6db9-49a0-8375-42374085c4f1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:58.981991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:58.982618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:58.982865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:58.983135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:58.983772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:58.985681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.014401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1177/2381] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:53:58 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:53:59.018340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.030392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe4f2-9016-4ac0-8360-b9b7b4f50950 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.104101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.104737 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.105101 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.105376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.106052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.108006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.118468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:59.135332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1205/2382] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:53:59 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:53:59.137240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.147869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd4b5-5e98-485f-a3e0-d186150e91ee admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.152628 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.153264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.153543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.153814 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.154479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.156175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.166812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1178/2383] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:53:59.170761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.181005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfbcad-1b7e-4a4e-956a-bf19b88916f9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.186687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.187374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.187726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.188082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.188790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.190517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996260-fbb2-4e52-b606-2c1b953814c3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.202141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1206/2384] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:53:59.206444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.218035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b046d0f-6c20-45ae-8ca7-063a447977cf admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.222803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.223473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.223757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.224031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] PATH_INFO: `/v3/projects/4b2635fe83b04b1d8ac63aadc7c28f22/users/c42e5e45489049e7bda60a010297fe10/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.239796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2635fe83b04b1d8ac63aadc7c28f22, user_id=c42e5e45489049e7bda60a010297fe10, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.242475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.258007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1179/2385] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:53:59 2022] PUT /identity/v3/projects/4b2635fe83b04b1d8ac63aadc7c28f22/users/c42e5e45489049e7bda60a010297fe10/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:59.269370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.269576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.269837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.300095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] MFA Rules not processed for user `c42e5e45489049e7bda60a010297fe10`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:59.301387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:59.372768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1207/2386] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:59.377602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.453810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace68bb6-97f7-4dc3-8c7b-fec7a46f36a7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.458647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.459397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.459684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.459973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.460632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.462605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.490457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1180/2387] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:53:59 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:53:59.494759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.507141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44f13a-9b2c-4086-a0fd-6387406c90bb None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.568654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.569307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.569593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.569863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.570559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.572511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.593890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1208/2388] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:53:59 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:53:59.598245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.610428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab25449-fa84-4434-a16a-902b5e27babd admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.615344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.616177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.616471 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.616745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.617431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.619015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.631272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1181/2389] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:53:59.635351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.645409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1970be-e556-4a03-a633-e0040c95fb67 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.650033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.650691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.650972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.651242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] PATH_INFO: `/v3/projects/7ba00beb1d694487b8078caa89ed4bdb/users/b46cc2a408e5436487fe46307ba54ad3/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.666089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba00beb1d694487b8078caa89ed4bdb, user_id=b46cc2a408e5436487fe46307ba54ad3, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.669346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.682485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1209/2390] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:53:59 2022] PUT /identity/v3/projects/7ba00beb1d694487b8078caa89ed4bdb/users/b46cc2a408e5436487fe46307ba54ad3/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:59.687482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.698498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade67e2c-b54c-4df0-bd5f-4014528f0da7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.724797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:59.762358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.762978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.763262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.763539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.764186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.765823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d43098-db5a-40c8-90cc-2bd7b7e87d66 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.779364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1182/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:53:59.780576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.790904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db60252-67ec-4f56-9445-ca391f3e9eb6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.796264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:59.796894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.797186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.797461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] PATH_INFO: `/v3/projects/7ba00beb1d694487b8078caa89ed4bdb/users/b46cc2a408e5436487fe46307ba54ad3/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.799349 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba00beb1d694487b8078caa89ed4bdb, user_id=b46cc2a408e5436487fe46307ba54ad3, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:59.802051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:59.813725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1210/2392] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:53:59 2022] PUT /identity/v3/projects/7ba00beb1d694487b8078caa89ed4bdb/users/b46cc2a408e5436487fe46307ba54ad3/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:53:59.825242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:59.825527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:59.825801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:59.853938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] MFA Rules not processed for user `b46cc2a408e5436487fe46307ba54ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:59.855644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:59.924556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1183/2393] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:53:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:59.936433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:59.938222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:59.965742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:59.995758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d731-c6e3-4bf6-9cbe-cf0d2a4b22d3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:59.999407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:00.000002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:00.000281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:00.000556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:00.009322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:00.010749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:00.020111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1211/2394] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:00.094202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:00.104083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829df70d-b130-454a-bc6c-90a910351e0a None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:00.108349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:00.109004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:00.109294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:00.109572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:00.164967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:00.166408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f61ca5-7f91-4c36-bf1b-a1d614a5b26e service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:00.175722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1184/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:10.015775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=15 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:10.171756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=79 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:45.600323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:45.701162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629142c-7d01-4d19-9788-ee8b016d2530 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:45.705650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:45.706266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:45.706543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:45.706812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] PATH_INFO: `/v3/users/c42e5e45489049e7bda60a010297fe10` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:45.707909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42e5e45489049e7bda60a010297fe10)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:45.709854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:45.759648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42e5e45489049e7bda60a010297fe10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:45.760281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42e5e45489049e7bda60a010297fe10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:45.771818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42e5e45489049e7bda60a010297fe10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:45.779103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42e5e45489049e7bda60a010297fe10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:45.786490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42e5e45489049e7bda60a010297fe10'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:45.793206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1212/2396] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:45 2022] DELETE /identity/v3/users/c42e5e45489049e7bda60a010297fe10 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:45.797681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:45.798203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:45.798302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:45.847425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:45.848654 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:45.910542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1185/2397] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:45.921257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:45.922660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:45.968665 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:46.024065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b458e8-dfa3-4f36-951d-35c2ee0f24bf admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:46.028228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:46.028834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:46.029155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:46.029433 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:46.038395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:46.039965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:46.049756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1213/2398] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:46.313432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:46.314881 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:46.327242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd547a-2bb0-4b58-89ba-8fede300c130 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:46.331905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:46.332541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:46.332828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:46.333143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] PATH_INFO: `/v3/projects/4b2635fe83b04b1d8ac63aadc7c28f22` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:46.334254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2635fe83b04b1d8ac63aadc7c28f22)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:46.336224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:46.382901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2635fe83b04b1d8ac63aadc7c28f22', 'request_id': 'req-1fa9dd31-247e-45c3-b60f-1f86946705b7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.390579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2635fe83b04b1d8ac63aadc7c28f22', 'request_id': 'req-1fa9dd31-247e-45c3-b60f-1f86946705b7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.396657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2635fe83b04b1d8ac63aadc7c28f22', 'request_id': 'req-1fa9dd31-247e-45c3-b60f-1f86946705b7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.406890 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] The token cache is being invalidate because project 4b2635fe83b04b1d8ac63aadc7c28f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:46.407170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.407798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.409313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1186/2399] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:46 2022] DELETE /identity/v3/projects/4b2635fe83b04b1d8ac63aadc7c28f22 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:46.413287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:46.423187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97a2c8-16d0-4bdb-a88d-97fc653b73d6 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:46.481774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:46.482319 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:46.482595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:46.482884 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] PATH_INFO: `/v3/users/b46cc2a408e5436487fe46307ba54ad3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:46.483983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46cc2a408e5436487fe46307ba54ad3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:46.486009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:46.530609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46cc2a408e5436487fe46307ba54ad3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.531328 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46cc2a408e5436487fe46307ba54ad3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.543272 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46cc2a408e5436487fe46307ba54ad3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.550067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46cc2a408e5436487fe46307ba54ad3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.557862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-5ee43852-860b-4e17-bdd0-c58af05e18d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46cc2a408e5436487fe46307ba54ad3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.564080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1214/2400] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:46 2022] DELETE /identity/v3/users/b46cc2a408e5436487fe46307ba54ad3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:46.827074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:46.828592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:46.845036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9dd31-247e-45c3-b60f-1f86946705b7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:46.895402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:46.903429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:46.904381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:46.904724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:46.905270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] PATH_INFO: `/v3/projects/7ba00beb1d694487b8078caa89ed4bdb` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:46.906483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba00beb1d694487b8078caa89ed4bdb)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:46.908574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:46.954403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba00beb1d694487b8078caa89ed4bdb', 'request_id': 'req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.961264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba00beb1d694487b8078caa89ed4bdb', 'request_id': 'req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.967334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba00beb1d694487b8078caa89ed4bdb', 'request_id': 'req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.978090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] The token cache is being invalidate because project 7ba00beb1d694487b8078caa89ed4bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:46.978379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.979010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-df81a47e-0d7f-4269-91a3-d4c3fe647c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:46.980461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1187/2401] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:46 2022] DELETE /identity/v3/projects/7ba00beb1d694487b8078caa89ed4bdb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:46.985383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:46.985866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:46.986050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.018564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:47.035742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:47.037037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:47.097639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1215/2402] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:47.107085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.107369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.107643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.127493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:47.128673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:47.136334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1188/2403] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:47.140416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.150574 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e63d3-585e-4445-be6d-c8f2129220dc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.155133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.155721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.156002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.156276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.156914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:47.157250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.158978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.171743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1216/2404] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:54:47.175943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.185467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae4172-c641-40fb-a95f-8cf329aaa444 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.190156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.190749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.191034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.191312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.191948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.193876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.221920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1189/2405] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:54:47.225952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.236326 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22f529-d9e1-4e06-bcdf-a84c73da7031 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.290692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.291212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.291518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.291792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.292409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.294332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.314416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1217/2406] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:47.318544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.328613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d146-0973-47c7-890f-235cb58b1f52 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.333223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.333819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.334103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.334376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.335009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.337146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.346920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1190/2407] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:47.350618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.361191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ed88-7023-46ff-8e74-39a8bc9cb1d0 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.365944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.366549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.366831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.367105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.367740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.369446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31e00d-c48e-45c8-9d39-eb3fdda347e3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.379921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1218/2408] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:47.384797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.394704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79460582-15b0-4b36-899f-8bd4d56d6d8a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.399779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.400423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.400704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.401005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] PATH_INFO: `/v3/projects/5069d547673a4545883fd1815748380f/users/57ebc21086dd4a858eb76354740070a4/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.415549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5069d547673a4545883fd1815748380f, user_id=57ebc21086dd4a858eb76354740070a4, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.418250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.431320 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1191/2409] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:47 2022] PUT /identity/v3/projects/5069d547673a4545883fd1815748380f/users/57ebc21086dd4a858eb76354740070a4/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:47.447423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.447710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.447987 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.475567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] MFA Rules not processed for user `57ebc21086dd4a858eb76354740070a4`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:47.476775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:47.530670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:47.540452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1219/2410] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:47.544904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.605989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1294de-4208-4338-9f3b-c3e0384960af admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.610523 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.611119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.611397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.611671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.612326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.614255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.639854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1192/2411] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:47.644246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.654727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2e0be-da4e-4c8b-850f-c26ecab0ad84 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.709931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.710532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.710814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.711088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.711749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.713716 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.734816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1220/2412] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:54:47.738943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.749877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d583e13-78eb-4034-9b6a-e245a300937a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.754504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.755124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.755391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.755671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.756305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.757920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.759963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:47.772212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1193/2413] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:47.773568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.784243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cc882-14e2-47aa-b394-cc273f3b317c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.789271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.790474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.790760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.791038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] PATH_INFO: `/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2/users/988699d10bc849f8aa515fe361ba1821/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.806472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32cbd71bb22540b5a8ebe4b20889f1a2, user_id=988699d10bc849f8aa515fe361ba1821, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.809123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.822129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1221/2414] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:47 2022] PUT /identity/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2/users/988699d10bc849f8aa515fe361ba1821/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:54:47.826218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.836350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc93cb6-c869-4558-8634-954758c5f697 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.893087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.893696 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.893982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.894259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.894897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.896480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64609f-aa5c-4acc-bfd5-77f7d7df6fdc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.906495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1194/2415] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:54:47.910709 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:47.921526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb88ce-1f6f-478c-8398-83e963cae6c8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:47.926056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:47.926698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.926977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.927248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] PATH_INFO: `/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2/users/988699d10bc849f8aa515fe361ba1821/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.929158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=32cbd71bb22540b5a8ebe4b20889f1a2, user_id=988699d10bc849f8aa515fe361ba1821, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:47.931834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:47.942115 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1222/2416] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:54:47 2022] PUT /identity/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2/users/988699d10bc849f8aa515fe361ba1821/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:47.951963 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:47.952260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:47.952731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:47.981578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] MFA Rules not processed for user `988699d10bc849f8aa515fe361ba1821`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:47.982826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:48.049903 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1195/2417] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:48.061609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:48.063374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:48.142943 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ee830-a16f-437a-96da-0bd66f9bb9fe admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:48.146652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:48.147257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:48.147540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:48.147815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:48.157210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:48.158642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:48.168511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1223/2418] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:48.243499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:48.253401 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50773-fd92-441a-8419-cedabb5b0e50 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:48.256968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:48.257568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:48.257846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:48.258116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:48.320381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:48.322319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769e077-24dc-43a2-8a23-d3c254e6f08a service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:48.331313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1196/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:58.163788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:58.327273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:58.514413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:58.574788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e36b5-1016-4317-98ac-a6524ee7f301 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:58.579331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:58.579890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:58.580164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:58.580435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] PATH_INFO: `/v3/users/57ebc21086dd4a858eb76354740070a4` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:58.581542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ebc21086dd4a858eb76354740070a4)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:58.583456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:58.626831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ebc21086dd4a858eb76354740070a4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:58.627468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ebc21086dd4a858eb76354740070a4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:58.639588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ebc21086dd4a858eb76354740070a4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:58.646277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ebc21086dd4a858eb76354740070a4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:58.653979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ebc21086dd4a858eb76354740070a4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:58.660082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1224/2420] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:58 2022] DELETE /identity/v3/users/57ebc21086dd4a858eb76354740070a4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:58.664896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:58.665220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:58.665530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:58.685440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:58.686631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:58.744435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1197/2421] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:58.754292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:58.755692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:58.817049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6d9b5-10df-4ff7-b108-6a19585dce00 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:58.820631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:58.821259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:58.821538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:58.821809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:58.830578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:58.831940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:58.841162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1225/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:59.109477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:59.110941 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:59.124560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da46d0b-9143-40fd-98ec-cbc54c9bd6a2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:59.129296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:59.130543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.130990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.131268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] PATH_INFO: `/v3/projects/5069d547673a4545883fd1815748380f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.132391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5069d547673a4545883fd1815748380f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:59.134314 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:59.180594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5069d547673a4545883fd1815748380f', 'request_id': 'req-e405f31a-c3b5-4ea8-adfa-b121e978d22d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.188162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5069d547673a4545883fd1815748380f', 'request_id': 'req-e405f31a-c3b5-4ea8-adfa-b121e978d22d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.194230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5069d547673a4545883fd1815748380f', 'request_id': 'req-e405f31a-c3b5-4ea8-adfa-b121e978d22d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.204871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] The token cache is being invalidate because project 5069d547673a4545883fd1815748380f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:59.205218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.205871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.207389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1198/2423] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:59 2022] DELETE /identity/v3/projects/5069d547673a4545883fd1815748380f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:59.212041 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:59.221583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644dd0-0146-432b-b970-4cbde15f865c service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:59.240239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:59.279885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:59.280508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.280795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.281103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] PATH_INFO: `/v3/users/988699d10bc849f8aa515fe361ba1821` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.282191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988699d10bc849f8aa515fe361ba1821)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:59.284141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:59.329664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988699d10bc849f8aa515fe361ba1821'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.330312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988699d10bc849f8aa515fe361ba1821'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.342786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988699d10bc849f8aa515fe361ba1821'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.349496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988699d10bc849f8aa515fe361ba1821'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.357560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-39bad0ce-0580-400d-b871-b20fddd27553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988699d10bc849f8aa515fe361ba1821'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.363702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1226/2424] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:54:59 2022] DELETE /identity/v3/users/988699d10bc849f8aa515fe361ba1821 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:59.646483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:59.648175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:59.668916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405f31a-c3b5-4ea8-adfa-b121e978d22d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:59.726885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:59.727520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.727833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.728115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] PATH_INFO: `/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.729270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32cbd71bb22540b5a8ebe4b20889f1a2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:59.731171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:59.760861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:59.778906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32cbd71bb22540b5a8ebe4b20889f1a2', 'request_id': 'req-1c73607f-b69f-43c3-b9ed-d652abeeca37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.785974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32cbd71bb22540b5a8ebe4b20889f1a2', 'request_id': 'req-1c73607f-b69f-43c3-b9ed-d652abeeca37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.792646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32cbd71bb22540b5a8ebe4b20889f1a2', 'request_id': 'req-1c73607f-b69f-43c3-b9ed-d652abeeca37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.802837 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] The token cache is being invalidate because project 32cbd71bb22540b5a8ebe4b20889f1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:54:59.803125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.803763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-1c73607f-b69f-43c3-b9ed-d652abeeca37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:54:59.805355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1199/2425] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:54:59 2022] DELETE /identity/v3/projects/32cbd71bb22540b5a8ebe4b20889f1a2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:54:59.811627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.811913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.812190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.833444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:59.834650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:59.893145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1227/2426] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:59.900929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.901349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.901653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.920835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:59.922057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:59.930287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1200/2427] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:59.934564 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:59.944193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86040152-b39c-40e4-be5b-86fdc9ec6820 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:59.949415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:59.950011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.950294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.950569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.951209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:54:59.951514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:59.953143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:59.964394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1228/2428] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:54:59.968484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:59.977706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788aab9-20e1-48d9-9dc9-a4171106fb53 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:59.982361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:59.982953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:59.983238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:59.983510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:59.984132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:59.986082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.013663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1201/2429] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:54:59 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:00.017656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.027510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f3ba0-0664-419e-843b-0a88813cd1f1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.052990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:00.085724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.088260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.088260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.088260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.088260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.089781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.115799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1229/2430] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:00 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:00.117093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.127671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dd3a8-2208-4a85-90fd-b352c2f3ff00 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.132267 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.133640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.135076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.135076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.135076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.136712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.147812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1202/2431] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:00.151672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.161798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56240a5-e702-427f-adfa-820827dceea9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.166454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.167043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.167324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.167597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.168228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.169898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f06543-2a95-4c7f-a9fb-7f54b5233c4a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.180932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1230/2432] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:00.185150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.200825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ff76-f7cf-4b8f-a36a-a572d215d562 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.205472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.206126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.206406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.206677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] PATH_INFO: `/v3/projects/de18e83733084c7284f1c69e03a94975/users/730976f7bd214539a766f6ae247f0cc4/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.221085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de18e83733084c7284f1c69e03a94975, user_id=730976f7bd214539a766f6ae247f0cc4, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.223784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.237191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1203/2433] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:00 2022] PUT /identity/v3/projects/de18e83733084c7284f1c69e03a94975/users/730976f7bd214539a766f6ae247f0cc4/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:00.247885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.248395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.248500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.276264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] MFA Rules not processed for user `730976f7bd214539a766f6ae247f0cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:00.277501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:00.339962 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1231/2434] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:00.344376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.406770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f202d-9249-47d5-afdc-64266e25dff9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.411331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.411932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.412217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.412493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.413152 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.415553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.442770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1204/2435] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:00 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:00.446854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.457712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151103-231c-425c-b048-ce3c143a2f9f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.514403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.515005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.515286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.515562 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.516180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.518156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.539523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1232/2436] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:00 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:00.543602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.553892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534787-8078-425a-91b3-13e0cbc24207 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.559597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.560192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.560476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.560753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.561434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.563019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.573395 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1205/2437] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:00.577311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.587347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4ba55-9602-4ee0-92ee-7352cd7adc99 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.591944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.592605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.592871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.593183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] PATH_INFO: `/v3/projects/f634c425ae7b420688112d5b4c6faedf/users/30eda6ac27fe4f14ad8b838d0d9d7802/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.608140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f634c425ae7b420688112d5b4c6faedf, user_id=30eda6ac27fe4f14ad8b838d0d9d7802, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.610854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.624358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1233/2438] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:00 2022] PUT /identity/v3/projects/f634c425ae7b420688112d5b4c6faedf/users/30eda6ac27fe4f14ad8b838d0d9d7802/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:00.628254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.638059 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eca16-f45a-48fb-a307-b74f2394d624 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.657880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:00.696347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.696948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.697265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.697537 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.698165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.699724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b1bf-50ec-477b-9dab-afe5d2f80436 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.709285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1206/2439] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:00.714112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.721076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:00.728370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007a7d-d018-4951-94b0-7c99badf3fc9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.733312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.733966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.734249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.734526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] PATH_INFO: `/v3/projects/f634c425ae7b420688112d5b4c6faedf/users/30eda6ac27fe4f14ad8b838d0d9d7802/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.736421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f634c425ae7b420688112d5b4c6faedf, user_id=30eda6ac27fe4f14ad8b838d0d9d7802, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.739108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.749154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1234/2440] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:00 2022] PUT /identity/v3/projects/f634c425ae7b420688112d5b4c6faedf/users/30eda6ac27fe4f14ad8b838d0d9d7802/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:00.760399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.760686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.760970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.790105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] MFA Rules not processed for user `30eda6ac27fe4f14ad8b838d0d9d7802`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:00.791298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:00.857985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1207/2441] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:00.868982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.870783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:00.927437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fb30e-86ee-4b62-b660-c4f6ae47b718 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.931128 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.931728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.932010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.932286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.941428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.942860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f3660-86bf-459f-99a5-a029c0e2d713 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.952141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1235/2442] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:01.018024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:01.079857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0768b0-c5e8-47be-9e87-f3c245760c61 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:01.084417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:01.085608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.086525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.086810 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] PATH_INFO: `/v3/users/730976f7bd214539a766f6ae247f0cc4` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.087942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=730976f7bd214539a766f6ae247f0cc4)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:01.089921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:01.135418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730976f7bd214539a766f6ae247f0cc4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.137085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:01.143250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730976f7bd214539a766f6ae247f0cc4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.155819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730976f7bd214539a766f6ae247f0cc4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.162785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730976f7bd214539a766f6ae247f0cc4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.163429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730976f7bd214539a766f6ae247f0cc4'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.169231 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1208/2443] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:01 2022] DELETE /identity/v3/users/730976f7bd214539a766f6ae247f0cc4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:01.173681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.173966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.174268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.193138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:01.194277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:01.252514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1236/2444] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:01.262176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:01.263582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:01.320792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa8f42-5269-4135-ae1f-956306e4b49c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:01.324477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:01.325130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.325414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.325689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.334558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:01.335964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:01.345350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1209/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:01.615991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:01.617650 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:01.630251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d9c1a-d61f-478a-8e86-501dab9bdb39 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:01.635279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:01.635918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.636198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.636467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] PATH_INFO: `/v3/projects/de18e83733084c7284f1c69e03a94975` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.637647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de18e83733084c7284f1c69e03a94975)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:01.639674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:01.687486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de18e83733084c7284f1c69e03a94975', 'request_id': 'req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.693071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de18e83733084c7284f1c69e03a94975', 'request_id': 'req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.699185 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de18e83733084c7284f1c69e03a94975', 'request_id': 'req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.711817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] The token cache is being invalidate because project de18e83733084c7284f1c69e03a94975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:01.712108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.712725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.714236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1237/2446] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:01 2022] DELETE /identity/v3/projects/de18e83733084c7284f1c69e03a94975 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:01.718308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:01.728383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb9765-1c40-4dc4-b861-d1bfb5dab3eb service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:01.785015 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:01.785647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.785931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.786209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] PATH_INFO: `/v3/users/30eda6ac27fe4f14ad8b838d0d9d7802` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.787324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30eda6ac27fe4f14ad8b838d0d9d7802)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:01.789251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:01.833857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30eda6ac27fe4f14ad8b838d0d9d7802'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.841407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30eda6ac27fe4f14ad8b838d0d9d7802'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.851932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:01.853150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30eda6ac27fe4f14ad8b838d0d9d7802'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.861388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30eda6ac27fe4f14ad8b838d0d9d7802'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.862134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0485da8a-82bb-463d-bf7f-220345bea6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30eda6ac27fe4f14ad8b838d0d9d7802'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.868699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1210/2447] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:01 2022] DELETE /identity/v3/users/30eda6ac27fe4f14ad8b838d0d9d7802 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:02.141933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.143363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:02.156305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bca61d-2e05-4f26-b4bf-6dfdf550f5a2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.170210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:02.217889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.218438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.218715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.218986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] PATH_INFO: `/v3/projects/f634c425ae7b420688112d5b4c6faedf` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.220076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f634c425ae7b420688112d5b4c6faedf)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.222025 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.266997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f634c425ae7b420688112d5b4c6faedf', 'request_id': 'req-60186755-7d70-4b23-b223-1e86f84d87b3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:02.272518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f634c425ae7b420688112d5b4c6faedf', 'request_id': 'req-60186755-7d70-4b23-b223-1e86f84d87b3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:02.278539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f634c425ae7b420688112d5b4c6faedf', 'request_id': 'req-60186755-7d70-4b23-b223-1e86f84d87b3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:02.290028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] The token cache is being invalidate because project f634c425ae7b420688112d5b4c6faedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:02.290325 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:02.290955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-60186755-7d70-4b23-b223-1e86f84d87b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:02.292418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1238/2448] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:02 2022] DELETE /identity/v3/projects/f634c425ae7b420688112d5b4c6faedf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:02.297742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.298032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.298312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.318304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:02.319500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:02.378030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1211/2449] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:02.385806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.386099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.386379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.406502 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:02.407695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:02.416046 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1239/2450] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:02.420181 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.429970 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cf55-1fb1-423e-bd30-3ae49771ac1d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.435220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.435792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.436155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.436502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.437192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:02.437495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.439149 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.450871 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1212/2451] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:02.455108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.464750 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6a54-b9bf-4668-b915-a831a6a2a0cd None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.470383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.470933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.471213 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.471485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.472119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.474340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.502338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1240/2452] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:02.506436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.516753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57a0c-4358-4d8f-87c8-6b116f135923 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.572542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.573239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.573524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.573802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.574435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.576344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.605691 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1213/2453] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:02.612290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.630714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ac632-9b55-4137-aa1a-12b9aee34313 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.636346 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.636970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.637267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.637542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.638179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.639791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.650911 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1241/2454] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:02.655080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.666307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b2b4-9cc8-4b8c-aeb2-eaf556fefd6a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.671221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.671905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.672296 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.673429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.673429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.675690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abc2e8-c6a5-4436-bbfb-e309d989b337 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.688751 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1214/2455] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:02.693023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.704484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7a8-bd65-4d88-9146-ec0d013ed454 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.710531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.711766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.712051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.712352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] PATH_INFO: `/v3/projects/08498537d44645d79eb8534375ed81b9/users/8de4b3482be445a3aaf8caa9992e6ac4/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.730872 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08498537d44645d79eb8534375ed81b9, user_id=8de4b3482be445a3aaf8caa9992e6ac4, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.734023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.750557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1242/2456] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:02 2022] PUT /identity/v3/projects/08498537d44645d79eb8534375ed81b9/users/8de4b3482be445a3aaf8caa9992e6ac4/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:02.761672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.762245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.762245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.791743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] MFA Rules not processed for user `8de4b3482be445a3aaf8caa9992e6ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:02.792975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:02.840350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:02.866443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1215/2457] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:02.870752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.921934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:02.942377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3fd54-ce63-4160-b0ad-e5d644ab72b8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:02.947438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:02.948039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:02.948318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:02.948586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:02.949254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:02.951167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:02.980384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1243/2458] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:02.984392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:02.995465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b271-3402-45b7-ad95-1983aafeec32 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.054596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.055213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.055492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.055767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.056404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.058390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.084248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1216/2459] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:02 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:03.088900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.101093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbda0ae-5e02-4181-84d6-45cef9b0a7ae admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.107277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.107896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.108168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.108450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.109132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.110740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.121514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1244/2460] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:03.125882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.136183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5272-07a6-4c32-a8fd-62c120926fde admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.141032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.141701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.141983 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.142257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] PATH_INFO: `/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25/users/1fc6c185800f4bb9864114c059cf650b/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.158099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e3557d3fda4dae8806f9dfb2308a25, user_id=1fc6c185800f4bb9864114c059cf650b, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.160752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.175888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1217/2461] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:03 2022] PUT /identity/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25/users/1fc6c185800f4bb9864114c059cf650b/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:03.180605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.192424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8ef8-2dc4-444b-9f79-0fc2f4894b3c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.250654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.251195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.251475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.251748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.252379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.254104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace53357-3607-4ca8-9cc9-e391a58ff965 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.268266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1245/2462] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:03.269467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.280163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76c31-48ee-4452-a0ce-c2828137d7d0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.285170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.285826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.286104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.286380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] PATH_INFO: `/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25/users/1fc6c185800f4bb9864114c059cf650b/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.288344 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e3557d3fda4dae8806f9dfb2308a25, user_id=1fc6c185800f4bb9864114c059cf650b, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.291475 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.302173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1218/2463] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:03 2022] PUT /identity/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25/users/1fc6c185800f4bb9864114c059cf650b/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:03.370663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.371290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.371578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.401488 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] MFA Rules not processed for user `1fc6c185800f4bb9864114c059cf650b`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:03.402721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:03.478804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1246/2464] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:03.491088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.492817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:03.550586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d20e5-247c-4609-89c1-2c66cfca63aa admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.554186 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.554862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.555143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.555418 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.564510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.565964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.576720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1219/2465] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:03.894154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.903555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14992363-14ae-4eba-94a2-cf6f0d7bdd39 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.908271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc6c185800f4bb9864114c059cf650b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7e3557d3fda4dae8806f9dfb2308a25', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.908877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.909188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.909469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.910144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:03.910455 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.912061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.923817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1247/2466] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:03.928132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.938200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4518ff-1897-41a3-a77b-340b95dd6511 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:03.943095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc6c185800f4bb9864114c059cf650b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7e3557d3fda4dae8806f9dfb2308a25', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:03.943702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:03.943991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:03.944273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:03.944917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:03.947421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:03.973979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1220/2467] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:03 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:03.978194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:03.988114 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95128bf-151a-4fbc-a56f-fdd84eb057be tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:04.009528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:04.047383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc6c185800f4bb9864114c059cf650b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7e3557d3fda4dae8806f9dfb2308a25', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:04.048094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:04.048380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:04.048654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:04.049316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:04.049619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:04.051302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:04.064174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1248/2468] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:04.104398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:04.114888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826d77c-8147-4e6a-abc6-cdda8617293c tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:04.120114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc6c185800f4bb9864114c059cf650b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7e3557d3fda4dae8806f9dfb2308a25', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:04.120748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:04.121063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:04.121467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] PATH_INFO: `/v3/projects/e3c5eefef11d4a808f4b1ba75c334649` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:04.122595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e3c5eefef11d4a808f4b1ba75c334649)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:04.124497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:04.135676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:04.172300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c5eefef11d4a808f4b1ba75c334649', 'request_id': 'req-7f97b86d-c039-4e86-8a9c-75a65e8eced1'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.180110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c5eefef11d4a808f4b1ba75c334649', 'request_id': 'req-7f97b86d-c039-4e86-8a9c-75a65e8eced1'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.187151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c5eefef11d4a808f4b1ba75c334649', 'request_id': 'req-7f97b86d-c039-4e86-8a9c-75a65e8eced1'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.198265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] The token cache is being invalidate because project e3c5eefef11d4a808f4b1ba75c334649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:04.198265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.199017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-7f97b86d-c039-4e86-8a9c-75a65e8eced1 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.201501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1221/2469] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:04 2022] DELETE /identity/v3/projects/e3c5eefef11d4a808f4b1ba75c334649 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:04.407260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:04.409022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:04.481143 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1da16d-868d-4f68-834d-645e44641761 tempest-VolumeQuotaClassesTest-1673877407 tempest-VolumeQuotaClassesTest-1673877407-project-admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:04.535821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:04.536445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:04.536726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:04.537029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] PATH_INFO: `/v3/users/8de4b3482be445a3aaf8caa9992e6ac4` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:04.538197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de4b3482be445a3aaf8caa9992e6ac4)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:04.540806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:04.586972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de4b3482be445a3aaf8caa9992e6ac4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.587558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de4b3482be445a3aaf8caa9992e6ac4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.599243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de4b3482be445a3aaf8caa9992e6ac4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.601217 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:04.607642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de4b3482be445a3aaf8caa9992e6ac4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.615132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de4b3482be445a3aaf8caa9992e6ac4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:04.621508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1249/2470] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:04 2022] DELETE /identity/v3/users/8de4b3482be445a3aaf8caa9992e6ac4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:04.626191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:04.626508 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:04.626789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:04.645707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:04.646887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:04.704605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1222/2471] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:04.714273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:04.715748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:04.772284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e65fb-da77-47d4-93fa-c194526fbaba admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:04.775927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:04.776521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:04.776801 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:04.777116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:04.786230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:04.787667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:04.797567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1250/2472] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:05.055846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.057336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:05.072887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b572-b71b-4442-987a-ed7095a5ee60 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:05.074485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:05.075125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.075474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.075673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] PATH_INFO: `/v3/projects/08498537d44645d79eb8534375ed81b9` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.076822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08498537d44645d79eb8534375ed81b9)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:05.078750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:05.126240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08498537d44645d79eb8534375ed81b9', 'request_id': 'req-9f1d1932-a678-4cf9-939a-147ff0162552'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.133862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08498537d44645d79eb8534375ed81b9', 'request_id': 'req-9f1d1932-a678-4cf9-939a-147ff0162552'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.140326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08498537d44645d79eb8534375ed81b9', 'request_id': 'req-9f1d1932-a678-4cf9-939a-147ff0162552'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.151161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] The token cache is being invalidate because project 08498537d44645d79eb8534375ed81b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:05.151451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.152082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.153605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1223/2473] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:05 2022] DELETE /identity/v3/projects/08498537d44645d79eb8534375ed81b9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:05.157689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.168017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41187f9-56cb-4b9d-b331-562795876c09 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:05.223432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:05.224054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.224331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.224604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] PATH_INFO: `/v3/users/1fc6c185800f4bb9864114c059cf650b` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.225726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc6c185800f4bb9864114c059cf650b)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:05.228222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:05.274510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc6c185800f4bb9864114c059cf650b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.275169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc6c185800f4bb9864114c059cf650b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.286973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc6c185800f4bb9864114c059cf650b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.293753 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc6c185800f4bb9864114c059cf650b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.300929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-00be3a49-d952-4f72-827a-19bebbfbbcbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc6c185800f4bb9864114c059cf650b'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.307206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1251/2474] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:05 2022] DELETE /identity/v3/users/1fc6c185800f4bb9864114c059cf650b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:05.584865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.586843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:05.600071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d1932-a678-4cf9-939a-147ff0162552 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:05.620293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:05.660717 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:05.661368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.661650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.661926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] PATH_INFO: `/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.663040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7e3557d3fda4dae8806f9dfb2308a25)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:05.664943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:05.710047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7e3557d3fda4dae8806f9dfb2308a25', 'request_id': 'req-54c4249b-df71-4650-9b21-28ae3a2ae398'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.718143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7e3557d3fda4dae8806f9dfb2308a25', 'request_id': 'req-54c4249b-df71-4650-9b21-28ae3a2ae398'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.724931 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7e3557d3fda4dae8806f9dfb2308a25', 'request_id': 'req-54c4249b-df71-4650-9b21-28ae3a2ae398'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.736372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] The token cache is being invalidate because project b7e3557d3fda4dae8806f9dfb2308a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:05.736671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.737363 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-54c4249b-df71-4650-9b21-28ae3a2ae398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:05.738828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1224/2475] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:05 2022] DELETE /identity/v3/projects/b7e3557d3fda4dae8806f9dfb2308a25 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:05.744066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.744542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.744648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.758463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:05.765479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:05.767588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:05.825682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1252/2476] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:05.833204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.833500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.833782 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.854558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:05.855778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:05.863649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1225/2477] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:05.868257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.877846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab99f8a-6ed9-483a-82a4-db1190d7fb5c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:05.882281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:05.883424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.883700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.883971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.884602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:05.884905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:05.886521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:05.897628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1253/2478] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:05.901612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.911220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1730-6070-4c1f-b4f3-84acc307f062 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:05.915835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:05.916436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:05.916724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:05.917025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:05.917662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:05.919568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:05.947252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1226/2479] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:05 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:05.951482 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:05.961249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7216752-1557-43a5-8849-6ac05fa763ca admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.016175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.016773 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.017075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.017368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.017990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.019901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.040291 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1254/2480] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:05 2022] POST /identity/v3/users => generated 296 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:06.044478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.055212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c12c-5ee7-45e2-9031-bfafb3ad14f5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.060026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.060634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.060885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.061188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.061809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.063504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.196692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1227/2481] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:06.201004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.211045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a567f-f140-4b5e-bda7-ea347a89bc3c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.215853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.216463 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.216747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.217060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.217727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.219337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eec9a6-ea9c-469d-b993-24209dea0eae admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.232315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1255/2482] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:06.237578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.248574 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a378d9-8565-4e9c-9058-97c0d1b1c133 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.253151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.253795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.254076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.254352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] PATH_INFO: `/v3/projects/4d7f85d1f1814490897ba250cd42bf9a/users/402b5f9e47b94d62ac85517e980e5f0d/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.269084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f85d1f1814490897ba250cd42bf9a, user_id=402b5f9e47b94d62ac85517e980e5f0d, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.271676 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.284688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1228/2483] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:06 2022] PUT /identity/v3/projects/4d7f85d1f1814490897ba250cd42bf9a/users/402b5f9e47b94d62ac85517e980e5f0d/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:06.295265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.295578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.295856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.323352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] MFA Rules not processed for user `402b5f9e47b94d62ac85517e980e5f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:06.324543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:06.388050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1256/2484] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:06.392548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.443062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:06.458388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdefc0-4ebd-41ea-babf-77f8a440cbb2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.463067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.463688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.464000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.464269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.464885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.466778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.493288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1229/2485] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:06 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:06.497359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.502747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:06.509905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902b72-a7f0-4a38-ba28-53148ea0b0f9 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.571532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.572165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.572450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.572728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.573389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.575318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.595783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1257/2486] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:06 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:06.599852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.610668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beab1c4-0a61-4705-b6d4-24ba0bac006e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.615349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.615984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.616265 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.616539 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.617813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.619386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.629247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1230/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:06.633265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.643098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cffdd-c1a3-432e-947b-be129aef44a4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.647635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.648774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.649087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.649361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] PATH_INFO: `/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92/users/1b3f118cc73449b48237401604be1adc/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.664376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1767f20be1ba40d6b7915b1fbee5bf92, user_id=1b3f118cc73449b48237401604be1adc, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.667096 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.681594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1258/2488] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:06 2022] PUT /identity/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92/users/1b3f118cc73449b48237401604be1adc/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:55:06.685921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.696121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f42b9-69d1-4ccf-86e1-4eef870fb301 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.751741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.752281 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.752561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.752832 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.753499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.755054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b720892-7a7f-4ad5-8768-2f9b36a3465a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.764645 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1231/2489] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:06.768781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.779796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cf8d-d586-4eb3-bc21-92e7b2dc80f4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.784339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.785009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.785293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.785566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] PATH_INFO: `/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92/users/1b3f118cc73449b48237401604be1adc/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.787448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1767f20be1ba40d6b7915b1fbee5bf92, user_id=1b3f118cc73449b48237401604be1adc, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:06.790692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:06.801003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1259/2490] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:06 2022] PUT /identity/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92/users/1b3f118cc73449b48237401604be1adc/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:06.812410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.812731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.813077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:06.840652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] MFA Rules not processed for user `1b3f118cc73449b48237401604be1adc`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:06.841910 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:06.916277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1232/2491] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:06.927049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:06.933330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:06.990196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3d08b-422b-492a-848d-0c3b1ce1c3ff admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:06.993864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:06.994459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:06.994739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:06.995013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:07.003858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:07.005238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:07.014898 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1260/2492] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:06 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:07.061076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:07.062521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:07.074948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92149cd6-782d-4da8-b84c-356205bfd412 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:07.078564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:07.079166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:07.079446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:07.079715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:07.121188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:07.137085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:07.138514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02efd97-28d4-4225-9d2a-3a9bee53055f service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:07.147141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1233/2493] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:10.440539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:10.497929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:10.502280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf30b7-7b78-4577-be54-79d2e9f05529 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:10.507670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:10.508288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:10.508569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:10.508843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] PATH_INFO: `/v3/users/402b5f9e47b94d62ac85517e980e5f0d` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:10.509990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402b5f9e47b94d62ac85517e980e5f0d)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:10.511915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:10.560266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402b5f9e47b94d62ac85517e980e5f0d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:10.560933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402b5f9e47b94d62ac85517e980e5f0d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:10.573279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402b5f9e47b94d62ac85517e980e5f0d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:10.580293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402b5f9e47b94d62ac85517e980e5f0d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:10.588018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402b5f9e47b94d62ac85517e980e5f0d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:10.593891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1261/2494] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:10 2022] DELETE /identity/v3/users/402b5f9e47b94d62ac85517e980e5f0d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:10.598376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:10.598827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:10.598921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:10.618169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:10.619302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:10.676761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1234/2495] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:10.686684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:10.688110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:10.744227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdcfdb-bec0-4137-a69f-0e7785f4e4d2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:10.747848 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:10.748465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:10.748727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:10.749029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:10.758225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:10.759654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:10.768840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1262/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:11.031094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.032494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:11.044424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefb0fe-5307-47ae-84a4-9d476af6c45f None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.049008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.049643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.049918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.050189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] PATH_INFO: `/v3/projects/4d7f85d1f1814490897ba250cd42bf9a` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.051315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7f85d1f1814490897ba250cd42bf9a)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:11.053223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:11.104878 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7f85d1f1814490897ba250cd42bf9a', 'request_id': 'req-fe014e19-3e44-4e06-9c83-3d18cddc9721'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.112351 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7f85d1f1814490897ba250cd42bf9a', 'request_id': 'req-fe014e19-3e44-4e06-9c83-3d18cddc9721'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.118979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7f85d1f1814490897ba250cd42bf9a', 'request_id': 'req-fe014e19-3e44-4e06-9c83-3d18cddc9721'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.128858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] The token cache is being invalidate because project 4d7f85d1f1814490897ba250cd42bf9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:11.129170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.129794 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.131237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1235/2497] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:11 2022] DELETE /identity/v3/projects/4d7f85d1f1814490897ba250cd42bf9a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:11.135415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.145249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f10638-fbc8-41dd-bb0c-f03b7c85b61a service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.202132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.202760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.203043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.203321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] PATH_INFO: `/v3/users/1b3f118cc73449b48237401604be1adc` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.204429 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3f118cc73449b48237401604be1adc)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:11.206395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:11.228476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:11.253074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3f118cc73449b48237401604be1adc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.253715 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3f118cc73449b48237401604be1adc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.265496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3f118cc73449b48237401604be1adc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.272522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3f118cc73449b48237401604be1adc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.280024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7b6f3599-5714-4644-8696-7822b38aaabb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3f118cc73449b48237401604be1adc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.285890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1263/2498] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:11 2022] DELETE /identity/v3/users/1b3f118cc73449b48237401604be1adc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:11.561045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.562603 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:11.582919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe014e19-3e44-4e06-9c83-3d18cddc9721 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.641198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.641928 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.642177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.642442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] PATH_INFO: `/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.643542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1767f20be1ba40d6b7915b1fbee5bf92)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:11.645436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:11.653620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:11.691407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1767f20be1ba40d6b7915b1fbee5bf92', 'request_id': 'req-f0bb7628-a019-403d-a74f-440287c15f5a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.699278 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1767f20be1ba40d6b7915b1fbee5bf92', 'request_id': 'req-f0bb7628-a019-403d-a74f-440287c15f5a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.705649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1767f20be1ba40d6b7915b1fbee5bf92', 'request_id': 'req-f0bb7628-a019-403d-a74f-440287c15f5a'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.716486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] The token cache is being invalidate because project 1767f20be1ba40d6b7915b1fbee5bf92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:11.716718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.717417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f0bb7628-a019-403d-a74f-440287c15f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:11.718927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1236/2499] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:11 2022] DELETE /identity/v3/projects/1767f20be1ba40d6b7915b1fbee5bf92 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:11.723853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.724359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.724458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.743701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:11.744909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:11.803331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1264/2500] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:11.811275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.811557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.811828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.831467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:11.832655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:11.841084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1237/2501] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:11.845268 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.856487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6add7-b0b7-4907-a1c5-ad9b68a41196 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.861790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.862412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.862698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.862975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.863714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:11.864030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:11.865745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:11.879083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1265/2502] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:11.884188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.893479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060158-751d-45ad-bd0d-25a77dd6d190 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.898098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.898687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.898962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.899230 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.899840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:11.901743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:11.930911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1238/2503] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:11 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:11.932028 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:11.942126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c458d-3787-48f5-a8e8-fc2ec0792791 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:11.996594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:11.997139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:11.997419 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:11.997689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:11.998314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.000210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.021671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1266/2504] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:11 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:55:12.025911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.036034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5ec3-4a80-4cc3-aeaa-997496f0f313 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.040602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.041223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.041503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.041776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.042407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.043998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.053619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1239/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:12.057999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.068010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4c95-ada2-4a5e-9b26-a802d27696e1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.073764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.074385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.074670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.075057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.075749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.077469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb9b2f-b99e-4972-b75c-23cc4d8b83aa admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.088356 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1267/2506] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:55:12.092685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.103075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c696fb9-4a1d-4116-93be-14d8db1e0e2f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.107835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.108484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.108766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.109103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] PATH_INFO: `/v3/projects/e39b55ab8579480cbd4f4aeb19093bbd/users/11be21caaacc4542baf070a4fbb1938c/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.124010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39b55ab8579480cbd4f4aeb19093bbd, user_id=11be21caaacc4542baf070a4fbb1938c, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.126673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.141300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1240/2507] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:12 2022] PUT /identity/v3/projects/e39b55ab8579480cbd4f4aeb19093bbd/users/11be21caaacc4542baf070a4fbb1938c/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:12.153234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.153524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.153798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.161800 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:12.183353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] MFA Rules not processed for user `11be21caaacc4542baf070a4fbb1938c`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:12.184819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:12.247804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1268/2508] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:12.259391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.261089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:12.320166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205294a1-0c44-43f4-b309-16d757c34241 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.323788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.324389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.324671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.324944 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.333062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.334467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96093a7-6371-42b8-9828-950ce79ad43d service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.343053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1241/2509] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:12.364876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.426913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8537d6-b14c-498e-a20f-05bdddb14416 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.431982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.432611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.432895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.433228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] PATH_INFO: `/v3/users/11be21caaacc4542baf070a4fbb1938c` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.434626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11be21caaacc4542baf070a4fbb1938c)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.436729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.480975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11be21caaacc4542baf070a4fbb1938c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:12.481661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11be21caaacc4542baf070a4fbb1938c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:12.493789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11be21caaacc4542baf070a4fbb1938c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:12.500550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11be21caaacc4542baf070a4fbb1938c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:12.508533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11be21caaacc4542baf070a4fbb1938c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:12.514405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1269/2510] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:12 2022] DELETE /identity/v3/users/11be21caaacc4542baf070a4fbb1938c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:12.518846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.519150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.519408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.538422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:12.539614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:12.568693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:12.599150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1242/2511] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:12.608865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.610322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:12.625074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:12.671440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd512ea8-3e85-49ba-b790-8ba48981f65e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.675059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:12.675662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:12.675941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:12.676215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:12.685359 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:12.686793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4cdf-507b-44e5-a7b8-ae740fcc423d service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:12.696061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1270/2512] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:12.971444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:12.974906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:12.994121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc81fad-515b-4da6-a36c-7e1527842009 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:12.999720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.000499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.000896 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.001443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] PATH_INFO: `/v3/projects/e39b55ab8579480cbd4f4aeb19093bbd` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.002629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39b55ab8579480cbd4f4aeb19093bbd)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.004608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.055867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39b55ab8579480cbd4f4aeb19093bbd', 'request_id': 'req-a0d25de3-e35e-47dd-a2b4-71328b636e6d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:13.063793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39b55ab8579480cbd4f4aeb19093bbd', 'request_id': 'req-a0d25de3-e35e-47dd-a2b4-71328b636e6d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:13.071504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39b55ab8579480cbd4f4aeb19093bbd', 'request_id': 'req-a0d25de3-e35e-47dd-a2b4-71328b636e6d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:13.083549 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] The token cache is being invalidate because project e39b55ab8579480cbd4f4aeb19093bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:13.083836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:13.084501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-a0d25de3-e35e-47dd-a2b4-71328b636e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:13.086855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1243/2513] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:12 2022] DELETE /identity/v3/projects/e39b55ab8579480cbd4f4aeb19093bbd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:13.091476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.091909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.092009 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.111341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:13.112570 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:13.175752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1271/2514] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:13.184169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.184459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.184732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.209229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:13.209229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:13.221417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1244/2515] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:13.225813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.235348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1d79a-1bbd-4c8f-9658-8c300b5ff33f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.239841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.240444 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.240724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.241023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.241683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:55:13.241964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.243839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.260383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1272/2516] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:55:13.262710 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.278112 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4f532-9b0a-4faa-a2c2-aec78d6fa3c1 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.282874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.283486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.283763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.284036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.285091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.288649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.324003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1245/2517] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:55:13 2022] POST /identity/v3/projects => generated 395 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:13.328534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.339689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2746be-0bf2-4fe0-840f-f7c1f695a51b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.395110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.395706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.395988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.396263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.396883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.398856 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.419620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1273/2518] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:55:13 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:55:13.423732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.434975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c78b5-612e-4ddb-bf6e-9838a89d9726 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.440253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.440853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.441168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.441445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.442079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.443653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.454068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1246/2519] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:13.457946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.467676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dda4c-8b57-40a2-ab41-46154f6a6b6c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.472320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.472906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.473212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.473484 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.474107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.475957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9f1e2-feba-4364-a2a6-3ad9d6dfa7ab admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.486109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1274/2520] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:55:13.489979 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.500192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c83315-d788-4dde-9fca-022edc6234dc admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.504747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.505422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.505705 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.506011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] PATH_INFO: `/v3/projects/1fa5be60094a4d6da8792b5b552f5846/users/b12ef6cc3d8a47ed9aab1b3fa77707a3/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.521942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa5be60094a4d6da8792b5b552f5846, user_id=b12ef6cc3d8a47ed9aab1b3fa77707a3, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.524668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.537246 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1247/2521] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:55:13 2022] PUT /identity/v3/projects/1fa5be60094a4d6da8792b5b552f5846/users/b12ef6cc3d8a47ed9aab1b3fa77707a3/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:13.547569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.547857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.548133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.576510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] MFA Rules not processed for user `b12ef6cc3d8a47ed9aab1b3fa77707a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:13.577767 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:13.587067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:13.646548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1275/2522] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:13.660704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:13.662441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:13.718850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e9d8-f078-4a3b-ae24-3a39cd03edd7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:13.722414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:13.723013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:13.723291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:13.723559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:13.731703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:13.733142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:13.736530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:13.743891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1248/2523] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:13 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:22.491894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:22.493350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:22.582111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eb585-3af9-4584-bb23-628b35d40e8a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:22.587605 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:22.588212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.588673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.588870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.590618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:22.602421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:22.603990 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb224-663f-4156-a393-e409fa384d08 service placement] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:22.613537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1276/2524] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:22.840818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:22.905208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f172ab-dad1-49ca-8e22-edd78baf4903 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:22.910358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:22.911055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:22.911692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:22.912006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] PATH_INFO: `/v3/users/b12ef6cc3d8a47ed9aab1b3fa77707a3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:22.914422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12ef6cc3d8a47ed9aab1b3fa77707a3)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:22.916510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:22.967262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12ef6cc3d8a47ed9aab1b3fa77707a3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.975335 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12ef6cc3d8a47ed9aab1b3fa77707a3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.988548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12ef6cc3d8a47ed9aab1b3fa77707a3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.996017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12ef6cc3d8a47ed9aab1b3fa77707a3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:22.996634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12ef6cc3d8a47ed9aab1b3fa77707a3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.002783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1249/2525] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:22 2022] DELETE /identity/v3/users/b12ef6cc3d8a47ed9aab1b3fa77707a3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:23.007781 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.008080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.008383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.028340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:23.029585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:23.089978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1277/2526] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:23.099600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.101081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:23.161223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4e9c-0878-4592-8807-9c7b961d4b73 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.165611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.166222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.166502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.166779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.175788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.177228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453aa31-3281-49a5-b1af-7a9d1defce60 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.187504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1250/2527] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:23.448111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:23.449634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:23.461722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b91f-744d-4f73-bb17-71cd411c75e0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:23.466462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:23.467087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:23.467363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:23.467633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] PATH_INFO: `/v3/projects/1fa5be60094a4d6da8792b5b552f5846` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:23.468734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa5be60094a4d6da8792b5b552f5846)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:23.470666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:23.487510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:23.517723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa5be60094a4d6da8792b5b552f5846', 'request_id': 'req-8f86346c-f73a-41f4-b417-6d2988847bb2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.523251 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa5be60094a4d6da8792b5b552f5846', 'request_id': 'req-8f86346c-f73a-41f4-b417-6d2988847bb2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.529385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa5be60094a4d6da8792b5b552f5846', 'request_id': 'req-8f86346c-f73a-41f4-b417-6d2988847bb2'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.541707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] The token cache is being invalidate because project 1fa5be60094a4d6da8792b5b552f5846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:23.542021 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.542643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8f86346c-f73a-41f4-b417-6d2988847bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:23.544176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1278/2528] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:23 2022] DELETE /identity/v3/projects/1fa5be60094a4d6da8792b5b552f5846 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:33.182905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:33.538815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:37.077425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:37.077947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:37.078053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:37.097713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] MFA Rules not processed for user `0fca332203b9473fa8b07286bc44300f`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:37.098915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:37.152627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1251/2529] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:55:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:47.146588 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=25 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:47.353518 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3939c5-cb5c-4247-9e99-ed7026b7f899 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.353993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3939c5-cb5c-4247-9e99-ed7026b7f899 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.354103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3939c5-cb5c-4247-9e99-ed7026b7f899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.382984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2d3939c5-cb5c-4247-9e99-ed7026b7f899 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:47.384186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2d3939c5-cb5c-4247-9e99-ed7026b7f899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.448870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1279/2530] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:55:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:55:47.496509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:47.497939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:47.565378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb68345-cd9d-45d6-b436-ec9e040652e2 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:47.570753 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:47.571354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:47.571626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:47.571893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:47.579829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:47.581416 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:47.589774 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1252/2531] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:55:57.442551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=32 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:57.586688 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=32 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:03.458565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:03.459059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:03.459167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:03.478209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:03.479383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:03.486396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1280/2532] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:56:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:03.498876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:03.509385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4609b-b51b-4c9a-a273-0ab2c30ab4dc service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:03.514429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:03.515021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:03.515305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:03.515582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:03.523481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:03.525076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:03.533028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1253/2533] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:56:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:56:11.891487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.893152 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.960219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071717ca-dbdd-4380-85e6-ff6e0925f80e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.963887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.964651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.964940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.965308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.967171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:12.028895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:12.030395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:12.042820 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1281/2534] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:13.529877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:22.036211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=51 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:24.339712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:24.341287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:24.425902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0bfdf-9e53-4630-9b38-b0eee6d3f3a5 service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:24.430226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:24.430850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:24.431167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:24.431476 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:24.433951 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:24.543228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:24.544590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795695aa-2726-4d58-afd5-70c1a2526906 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:24.568921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1254/2535] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:26.929002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:26.930440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:26.942516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448e35-0ae7-471d-b796-fedaa2422708 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:26.946163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:26.946763 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:26.947047 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:26.947323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:26.949142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:26.963833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:26.965300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2904f5-8764-493f-8c4d-574b56bf6c0e service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:26.977082 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1282/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:27.259147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:27.259648 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:27.259733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:27.278800 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] MFA Rules not processed for user `5669f5e5dcd74e01b4fb7c1bd512bd82`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:27.279982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:27.286757 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1255/2537] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:28.515803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:28.516245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:28.516347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:28.535187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:28.536383 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:28.543386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1283/2538] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 09:56:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:28.555850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:28.568495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12a9cf-c597-47a0-922c-693a39c5db78 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:28.573466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:28.574187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:28.574470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:28.574761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:28.582627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:28.584220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:28.592066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1256/2539] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:56:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:56:38.529588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=9 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:38.588875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=72 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:48.854967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:48.869583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3936b-a967-4a6f-9459-308c9f9fe14a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:48.873631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:48.874680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:48.874680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:48.874992 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:48.877744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:48.889660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:48.891677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:48.899585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1284/2540] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:51.458270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:51.467884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507a6ce-d6f0-44ae-945b-c92bc822982d service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:51.471792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:51.472342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:51.472621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:51.472980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:51.474669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:51.539862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:51.541331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:51.551559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1257/2541] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:52.052687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:52.061651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762b4c-db91-4779-8e1b-7c6226b49af3 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:52.065755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:52.066352 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:52.066632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:52.066903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:52.072786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:52.135955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:52.137569 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:52.147443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1285/2542] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:54.954056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:54.955797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:54.982811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c6c42-0f99-4dec-a6db-96b61460e1a7 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:54.990205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:54.990875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:54.991158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:54.991439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] PATH_INFO: `/v3/users/a0101b6f23d04016a89c986621ba5dde` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.009531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0101b6f23d04016a89c986621ba5dde)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.011459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.067876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0101b6f23d04016a89c986621ba5dde'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.077000 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0101b6f23d04016a89c986621ba5dde'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.093313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0101b6f23d04016a89c986621ba5dde'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.102378 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0101b6f23d04016a89c986621ba5dde'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.103384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b79f8f2e-5d67-4070-9454-fe2d0a474e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0101b6f23d04016a89c986621ba5dde'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.109835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1258/2543] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:54 2022] DELETE /identity/v3/users/a0101b6f23d04016a89c986621ba5dde => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:55.285097 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.297173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e9542-0c38-41aa-9ff8-04306304d1c3 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.357443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.357999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.358316 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.358594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] PATH_INFO: `/v3/projects/3fd1bef8389445f0a8cfe492cbf22f8c` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.369545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd1bef8389445f0a8cfe492cbf22f8c)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.371472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.422220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd1bef8389445f0a8cfe492cbf22f8c', 'request_id': 'req-db97ad45-3488-40b5-8ad5-cf8692c082d3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.428449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd1bef8389445f0a8cfe492cbf22f8c', 'request_id': 'req-db97ad45-3488-40b5-8ad5-cf8692c082d3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.435862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd1bef8389445f0a8cfe492cbf22f8c', 'request_id': 'req-db97ad45-3488-40b5-8ad5-cf8692c082d3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.448018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] The token cache is being invalidate because project 3fd1bef8389445f0a8cfe492cbf22f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:55.448305 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.448915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-db97ad45-3488-40b5-8ad5-cf8692c082d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:55.450405 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1286/2544] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:55 2022] DELETE /identity/v3/projects/3fd1bef8389445f0a8cfe492cbf22f8c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:55.455745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.456113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.456439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.476866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:55.478097 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:55.525677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:55.538749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1259/2545] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:55.547427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.547714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.547989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.558055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:55.573091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:55.574377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:55.582263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1287/2546] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:55.586525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.596594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a7f4c-f1c1-49f9-8fb8-12bc58053fc9 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.601129 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.601724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.602006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.602277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.602916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:55.603214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.604819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.616980 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1260/2547] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:56:55.621095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.630266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c6a87-1e0f-47e2-bb20-26ddd9741b8b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.634931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.635523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.635803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.636077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.636695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.638627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.674655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1288/2548] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:55 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:55.678938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.689437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a7a7-7a97-42ea-ba18-f7cf31589666 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.745109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.745712 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.745996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.746271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.746902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.748792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.770433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1261/2549] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:55 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:56:55.774787 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.785883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7271206-73e7-478e-8654-c533f8329a1a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.790660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.791264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.791551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.791828 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.792461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.794145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.804457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1289/2550] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:55.808544 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.818748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6f081-31bf-4603-bb55-45bc3bea158b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.823388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.823982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.824264 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.824541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.825212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.826818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8acedd-c8d9-42c9-8fd2-d50fa232f696 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.837820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1262/2551] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:55.842037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:55.851968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41342cde-d81a-4a84-b23f-465982f847b3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:55.856608 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:55.857308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.857595 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.857873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] PATH_INFO: `/v3/projects/98e095a935ef476ca79e95b4baea91ad/users/9c4b6c39117946659b2630e990468df3/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.872921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e095a935ef476ca79e95b4baea91ad, user_id=9c4b6c39117946659b2630e990468df3, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:55.875554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:55.889755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1290/2552] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:55 2022] PUT /identity/v3/projects/98e095a935ef476ca79e95b4baea91ad/users/9c4b6c39117946659b2630e990468df3/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:55.899563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:55.899927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:55.900235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:55.927511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] MFA Rules not processed for user `9c4b6c39117946659b2630e990468df3`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:55.928726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:55.990168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1263/2553] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:56.002374 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:56.004066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:56.077555 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d33e2f-eda6-43ce-8a2a-58506c12d8a8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:56.083334 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:56.084010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:56.084304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:56.084632 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:56.092674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:56.094338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c1f8-791e-4c0a-bed7-0bca4c011092 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:56.103042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1291/2554] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:58.784759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:58.833560 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:58.846136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f466d-073c-4c05-8a3d-5eb6f08fc095 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:58.850624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:58.851288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:58.851571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:58.851850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] PATH_INFO: `/v3/users/9c4b6c39117946659b2630e990468df3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:58.852997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4b6c39117946659b2630e990468df3)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:58.854907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:58.900732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4b6c39117946659b2630e990468df3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:58.907516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4b6c39117946659b2630e990468df3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:58.919257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4b6c39117946659b2630e990468df3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:58.926610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4b6c39117946659b2630e990468df3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:58.927259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4b6c39117946659b2630e990468df3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:58.933415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1264/2555] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:58 2022] DELETE /identity/v3/users/9c4b6c39117946659b2630e990468df3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:58.938064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:58.938607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:58.939207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:58.957609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:58.958811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:59.016212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1292/2556] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:59.026375 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.027761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:59.091666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa2da-b5e9-4e41-bd33-364145d2f3b2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.095417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.096021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.096299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.096571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.105576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.106994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7c2e-c649-4917-96ea-1522ca363ee2 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.116947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1265/2557] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:59.384304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.386000 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:59.399806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc15f3-c5bf-416c-993b-a7321cd5790e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.404302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.405004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.405790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.406075 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] PATH_INFO: `/v3/projects/98e095a935ef476ca79e95b4baea91ad` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.407933 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e095a935ef476ca79e95b4baea91ad)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.409860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.420881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:59.458681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e095a935ef476ca79e95b4baea91ad', 'request_id': 'req-67a4c0b3-524a-4068-b76f-7fd5e322cc93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:59.464729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e095a935ef476ca79e95b4baea91ad', 'request_id': 'req-67a4c0b3-524a-4068-b76f-7fd5e322cc93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:59.472270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e095a935ef476ca79e95b4baea91ad', 'request_id': 'req-67a4c0b3-524a-4068-b76f-7fd5e322cc93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:59.483689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] The token cache is being invalidate because project 98e095a935ef476ca79e95b4baea91ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:59.483976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:59.484702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-67a4c0b3-524a-4068-b76f-7fd5e322cc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:59.486803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1293/2558] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:59 2022] DELETE /identity/v3/projects/98e095a935ef476ca79e95b4baea91ad => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:59.491754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.492050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.492334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.511975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:59.513183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:59.575139 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1266/2559] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:59.585819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.586113 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.586395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.607731 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:59.609089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:59.621056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1294/2560] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:59.626833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.636349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8661ce-385c-4c09-93e4-015d8c722e84 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.640879 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.641503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.641781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.642052 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.642685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:59.642984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.644552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.652901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:59.656797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1267/2561] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:59.661471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.670746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa2201-c56d-45b5-b5bb-e62d45e68706 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.676536 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.677234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.677522 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.677799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.678426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.680338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.707424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1295/2562] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:59 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:59.711640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.721409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399bb54-4623-4891-8f8a-c16a7fca222d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.775856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.776483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.776761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.777060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.777666 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.779592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.800447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1268/2563] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:59 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:59.804397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.815643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685696b-27d8-4008-baea-7ca265892a1e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.820238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.820834 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.821160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.821441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.822074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.823668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.834144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1296/2564] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:59.837912 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.848233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411c7bd-89d0-4b20-8ca5-ac53dff25702 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.853049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.853643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.853926 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.854204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.854865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.856439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbe966-6e79-4e7c-9b3a-cf3581268694 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.866924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1269/2565] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:59.870764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:59.880571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6403bc-7cec-484f-bb53-7e8b7db634f9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:59.885287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:59.885940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.886224 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.886504 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] PATH_INFO: `/v3/projects/22d20ce95e0b40708e19c29cf22d74fb/users/507238fbab8e481894f064f88b835554/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.901937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d20ce95e0b40708e19c29cf22d74fb, user_id=507238fbab8e481894f064f88b835554, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:59.904685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:59.917849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1297/2566] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:59 2022] PUT /identity/v3/projects/22d20ce95e0b40708e19c29cf22d74fb/users/507238fbab8e481894f064f88b835554/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:59.929192 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:59.929483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:59.929760 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:59.958039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] MFA Rules not processed for user `507238fbab8e481894f064f88b835554`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:59.959244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:00.022383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1270/2567] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:56:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:00.026901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.089017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464d539-6ef4-41de-ba8b-ebb2028b1ab7 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.093565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.094169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.094450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.094777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.095404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.097331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.123347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1298/2568] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:00 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:00.127990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.138697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f5149-1f95-48db-a9a2-2f70952b1696 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.194651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.195259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.195536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.195805 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.196432 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.198370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.214372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:00.220859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1271/2569] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:00 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:00.224695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.234904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16b663-0bd5-414a-b853-9d3eda08b68f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.239450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.240045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.240345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.240601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.241266 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.242862 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.253659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1299/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:00.257700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.267575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccfcab-9a48-409c-af37-eae5f5bee441 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.272294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.272937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.273256 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.273536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] PATH_INFO: `/v3/projects/6bf29d77770444a390df1d31be35d69f/users/97359a8c876a42e58cb2a3807caeb2d1/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.288038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf29d77770444a390df1d31be35d69f, user_id=97359a8c876a42e58cb2a3807caeb2d1, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.291213 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.304348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1272/2571] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:00 2022] PUT /identity/v3/projects/6bf29d77770444a390df1d31be35d69f/users/97359a8c876a42e58cb2a3807caeb2d1/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:00.308524 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.318470 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7953a-255d-4540-b320-eb253f78200f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.323243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:00.384187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.384719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.385030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.385310 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.385940 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.387576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f38b0-ba27-44e2-b4a6-70b8b619242a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.397705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1300/2572] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:00.401863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.412585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190aa3-344c-4462-833e-75abe0f9b52a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.417193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.417836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.418117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.418388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] PATH_INFO: `/v3/projects/6bf29d77770444a390df1d31be35d69f/users/97359a8c876a42e58cb2a3807caeb2d1/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.420300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf29d77770444a390df1d31be35d69f, user_id=97359a8c876a42e58cb2a3807caeb2d1, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.422949 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.433176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1273/2573] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:00 2022] PUT /identity/v3/projects/6bf29d77770444a390df1d31be35d69f/users/97359a8c876a42e58cb2a3807caeb2d1/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:00.443105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.443398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.443672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.473439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] MFA Rules not processed for user `97359a8c876a42e58cb2a3807caeb2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:00.474642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:00.540493 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1301/2574] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:00.551357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:00.553083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:00.610512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0b92-ddc5-4324-9187-9142ce9bc495 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:00.614123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:00.614720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:00.615002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:00.615274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:00.629806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:00.632021 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1df19d-5614-4e5d-a450-886ee9c3f6d4 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:00.642441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1274/2575] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:01.961103 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.023252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d19a0-ac3e-41fd-afc4-ed1dffcfcdaf None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.027803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.028437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.028718 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.029018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] PATH_INFO: `/v3/users/507238fbab8e481894f064f88b835554` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.030165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=507238fbab8e481894f064f88b835554)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.032601 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.066824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.080118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507238fbab8e481894f064f88b835554'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.080914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507238fbab8e481894f064f88b835554'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.093999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507238fbab8e481894f064f88b835554'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.101288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507238fbab8e481894f064f88b835554'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.108576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507238fbab8e481894f064f88b835554'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.115221 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1302/2576] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:01 2022] DELETE /identity/v3/users/507238fbab8e481894f064f88b835554 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:02.119660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.120148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.120249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.138797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:02.139995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.198222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1275/2577] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:02.208657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.210109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.267055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796768-9fe5-4320-82d1-5a55947bd8dd admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.270805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.271408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.271694 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.271971 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.281019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.283013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.292435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1303/2578] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:02.565614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.567080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:02.580329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380669-e86d-4bc7-9a5c-ed4280fe9c04 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.585166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.586393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.586866 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.587203 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] PATH_INFO: `/v3/projects/22d20ce95e0b40708e19c29cf22d74fb` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.588298 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d20ce95e0b40708e19c29cf22d74fb)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.590254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.636083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d20ce95e0b40708e19c29cf22d74fb', 'request_id': 'req-e13d4182-a11e-4945-9997-44e4b46136ca'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.644587 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d20ce95e0b40708e19c29cf22d74fb', 'request_id': 'req-e13d4182-a11e-4945-9997-44e4b46136ca'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.646509 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.653221 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d20ce95e0b40708e19c29cf22d74fb', 'request_id': 'req-e13d4182-a11e-4945-9997-44e4b46136ca'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.672702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] The token cache is being invalidate because project 22d20ce95e0b40708e19c29cf22d74fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:02.673025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.673718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.675294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1276/2579] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:02 2022] DELETE /identity/v3/projects/22d20ce95e0b40708e19c29cf22d74fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:02.679507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:02.690379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87b1de-3bb7-4eb2-99c1-ac76f5354c49 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:02.745283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:02.745829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:02.746107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:02.746378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] PATH_INFO: `/v3/users/97359a8c876a42e58cb2a3807caeb2d1` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:02.747490 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97359a8c876a42e58cb2a3807caeb2d1)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:02.749521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:02.794643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97359a8c876a42e58cb2a3807caeb2d1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.795295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97359a8c876a42e58cb2a3807caeb2d1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.800418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:02.808282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97359a8c876a42e58cb2a3807caeb2d1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.816206 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97359a8c876a42e58cb2a3807caeb2d1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.823629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-a607b9bb-500c-4525-b7fe-8597337b0e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97359a8c876a42e58cb2a3807caeb2d1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:02.829976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1304/2580] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:02 2022] DELETE /identity/v3/users/97359a8c876a42e58cb2a3807caeb2d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:03.099372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.101080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:03.115752 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4182-a11e-4945-9997-44e4b46136ca admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.179531 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.180193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.180452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.180834 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] PATH_INFO: `/v3/projects/6bf29d77770444a390df1d31be35d69f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.182275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf29d77770444a390df1d31be35d69f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.184108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.236158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf29d77770444a390df1d31be35d69f', 'request_id': 'req-d322121e-15ed-4ee5-9b39-bc2d2ef12315'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:03.245448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf29d77770444a390df1d31be35d69f', 'request_id': 'req-d322121e-15ed-4ee5-9b39-bc2d2ef12315'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:03.252136 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf29d77770444a390df1d31be35d69f', 'request_id': 'req-d322121e-15ed-4ee5-9b39-bc2d2ef12315'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:03.263039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] The token cache is being invalidate because project 6bf29d77770444a390df1d31be35d69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:03.263331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:03.263988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-d322121e-15ed-4ee5-9b39-bc2d2ef12315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:03.265748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1277/2581] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:03 2022] DELETE /identity/v3/projects/6bf29d77770444a390df1d31be35d69f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:03.271060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.271532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.271629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.292867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:03.294126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:03.361170 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1305/2582] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:03.374004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.374293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.374573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.395495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:03.396686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:03.404552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1278/2583] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:03.408846 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.418435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcab1c-1047-4731-a6ed-b4dd7ebb6d3b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.423011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.423600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.423879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.424166 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.424793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:03.425145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.426741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.438720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1306/2584] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:03.442911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.452047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31485c20-4472-42f2-b1e1-d25467ddf32d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.458585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.459516 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.459939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.460220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.460853 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.462781 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.491420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1279/2585] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.495803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.506332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ba7a4-dab3-458f-bfbc-445bf7f29419 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.561271 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.561806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.562087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.562360 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.562984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.565471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.586489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1307/2586] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/users => generated 304 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.590765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.601307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8274b-ee62-4dd4-a99b-200d0b9ce955 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.605950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.606545 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.606827 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.607103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.607733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.609353 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.619140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1280/2587] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:03.623050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.636093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311fc74-4cd3-4c60-9f7b-b28425d20c86 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.641061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.642229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.642535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.642806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.643498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.645179 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce961521-61b7-4446-ad65-6f1d3e9fb6b4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.655921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1308/2588] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:03.660087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.669919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce927f1-4a61-4bee-82e0-46411424d565 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.674998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.675639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.675916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.676191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] PATH_INFO: `/v3/projects/823c742e4d734eee977a13fccbedc427/users/c3556b257fe84855bbe06789ae97b8f8/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.691819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=823c742e4d734eee977a13fccbedc427, user_id=c3556b257fe84855bbe06789ae97b8f8, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.694456 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.707792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1281/2589] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:03 2022] PUT /identity/v3/projects/823c742e4d734eee977a13fccbedc427/users/c3556b257fe84855bbe06789ae97b8f8/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:03.718145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.718430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.718704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.745817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] MFA Rules not processed for user `c3556b257fe84855bbe06789ae97b8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:03.747006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:03.772835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:03.811607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1309/2590] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:03.816100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.829917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:03.879730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975d2fd-197c-46e2-9d9a-710353fd6c6c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.884349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.884950 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.885261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.885536 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.886167 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.888037 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:03.914799 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1282/2591] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:03.918991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:03.929567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26447-ca44-4838-b35b-3f2f1f8ddf6b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:03.984008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:03.984624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:03.984880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:03.985205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:03.985829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:03.987735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.008307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1310/2592] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:03 2022] POST /identity/v3/users => generated 311 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:04.012317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.022997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a4ba-d2ca-4adf-86fc-0e39df5458ca admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.027553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.028130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.028413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.028690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.029364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.030930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.041856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1283/2593] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:04.046013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.056324 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27de6c3-1b4d-48ee-9b41-4129ba4698d3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.060944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.061682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.061913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.062190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] PATH_INFO: `/v3/projects/661520ee8afb420591ba61b623354581/users/a9e31a8d711a4a2dad6e7921fdf0f5ca/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.079136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661520ee8afb420591ba61b623354581, user_id=a9e31a8d711a4a2dad6e7921fdf0f5ca, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.081841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.096890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1311/2594] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:04 2022] PUT /identity/v3/projects/661520ee8afb420591ba61b623354581/users/a9e31a8d711a4a2dad6e7921fdf0f5ca/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:04.101017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.110989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8274aa-1476-496c-85ca-b66713e00a4e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.166301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.166895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.167173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.167444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.168072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.169671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05816954-6a10-4fb3-bfe7-abf0b8503268 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.179596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1284/2595] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:04.184004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.194373 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627259-93f8-4218-b1f5-b4248b656960 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.199098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.199747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.200027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.200300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] PATH_INFO: `/v3/projects/661520ee8afb420591ba61b623354581/users/a9e31a8d711a4a2dad6e7921fdf0f5ca/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.202236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=661520ee8afb420591ba61b623354581, user_id=a9e31a8d711a4a2dad6e7921fdf0f5ca, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.204871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.214795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1312/2596] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:04 2022] PUT /identity/v3/projects/661520ee8afb420591ba61b623354581/users/a9e31a8d711a4a2dad6e7921fdf0f5ca/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:04.225254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.225630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.225976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.254367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] MFA Rules not processed for user `a9e31a8d711a4a2dad6e7921fdf0f5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:04.255562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:04.323074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1285/2597] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:04 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:04.333667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.335368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:04.392317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bd04-ce92-44fb-b607-ecbc53f0bc4e admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.395938 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.397746 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.398019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.398283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.412090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.413626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.423505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1313/2598] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:04.572262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:04.582399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0505c1a-a033-44c4-8b63-73c4e0ab01f9 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:04.585978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:04.586580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:04.586863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:04.587168 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:04.592245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:04.646478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:04.647918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652cd0f-ba9a-4e98-a604-e506bdec0869 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:04.656711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1286/2599] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:04 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:14.153812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:14.179465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:14.218460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684af8c-323f-4737-a500-f064d1fd3795 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:14.223546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:14.224171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:14.224450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:14.224723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] PATH_INFO: `/v3/users/c3556b257fe84855bbe06789ae97b8f8` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:14.225892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3556b257fe84855bbe06789ae97b8f8)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:14.228007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:14.272320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3556b257fe84855bbe06789ae97b8f8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.272909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3556b257fe84855bbe06789ae97b8f8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.285172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3556b257fe84855bbe06789ae97b8f8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.291949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3556b257fe84855bbe06789ae97b8f8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.299201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3556b257fe84855bbe06789ae97b8f8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.305265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1314/2600] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:14 2022] DELETE /identity/v3/users/c3556b257fe84855bbe06789ae97b8f8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:14.309992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:14.310514 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:14.310619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:14.329783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:14.330967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:14.388606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1287/2601] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:14.398227 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:14.399738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:14.455836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e20f-0fd6-4d64-ac37-76a848ee1e0f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:14.459561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:14.460167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:14.460449 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:14.460723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:14.469791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:14.471225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:14.480446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1315/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:14.745945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:14.747388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:14.759978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85100eab-3756-4c46-8667-0e8f941c0780 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:14.764723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:14.765381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:14.765665 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:14.765973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] PATH_INFO: `/v3/projects/823c742e4d734eee977a13fccbedc427` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:14.767151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=823c742e4d734eee977a13fccbedc427)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:14.769008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:14.816982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823c742e4d734eee977a13fccbedc427', 'request_id': 'req-e11e4c08-4c20-40c0-ac6a-38da4e16f040'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.823961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823c742e4d734eee977a13fccbedc427', 'request_id': 'req-e11e4c08-4c20-40c0-ac6a-38da4e16f040'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.830508 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823c742e4d734eee977a13fccbedc427', 'request_id': 'req-e11e4c08-4c20-40c0-ac6a-38da4e16f040'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.840611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] The token cache is being invalidate because project 823c742e4d734eee977a13fccbedc427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:14.840904 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.841591 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.843065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1288/2603] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:14 2022] DELETE /identity/v3/projects/823c742e4d734eee977a13fccbedc427 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:14.848314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:14.858385 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5815e5-9967-4123-a443-cdd039077a62 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:14.899153 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:14.915529 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:14.916175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:14.916459 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:14.916738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] PATH_INFO: `/v3/users/a9e31a8d711a4a2dad6e7921fdf0f5ca` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:14.917893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e31a8d711a4a2dad6e7921fdf0f5ca)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:14.919823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:14.964197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e31a8d711a4a2dad6e7921fdf0f5ca'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.964845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e31a8d711a4a2dad6e7921fdf0f5ca'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.977015 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e31a8d711a4a2dad6e7921fdf0f5ca'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.983759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e31a8d711a4a2dad6e7921fdf0f5ca'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.990914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-862d4db7-caa7-471a-a26f-ed6973f842d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e31a8d711a4a2dad6e7921fdf0f5ca'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:14.996690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1316/2604] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:14 2022] DELETE /identity/v3/users/a9e31a8d711a4a2dad6e7921fdf0f5ca => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:15.262242 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.263720 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:15.278612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4c08-4c20-40c0-ac6a-38da4e16f040 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.309887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:15.338543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.339200 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.339480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.339702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] PATH_INFO: `/v3/projects/661520ee8afb420591ba61b623354581` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.340793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=661520ee8afb420591ba61b623354581)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.342682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.387094 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661520ee8afb420591ba61b623354581', 'request_id': 'req-99ae5713-bc15-407e-bee7-e200e60ebe95'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:15.394011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661520ee8afb420591ba61b623354581', 'request_id': 'req-99ae5713-bc15-407e-bee7-e200e60ebe95'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:15.400146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661520ee8afb420591ba61b623354581', 'request_id': 'req-99ae5713-bc15-407e-bee7-e200e60ebe95'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:15.410345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] The token cache is being invalidate because project 661520ee8afb420591ba61b623354581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:15.410635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:15.411290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-99ae5713-bc15-407e-bee7-e200e60ebe95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:15.412819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1289/2605] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:15 2022] DELETE /identity/v3/projects/661520ee8afb420591ba61b623354581 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:15.417647 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.418119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.418236 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.438048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:15.439231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:15.496086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1317/2606] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:15.505449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.505740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.506047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.526478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:15.527685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:15.535806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1290/2607] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:15.539997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.549547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3de23-6bd4-4dd7-b0ad-846c6f5cf329 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.554085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.554675 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.554952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.555222 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.556112 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:15.556407 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.558023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.569793 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1318/2608] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:15.573888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.584172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874e32b-5c59-47a8-89a2-11974366ec32 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.588872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.589552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.589778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.590046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.590669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.592535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.619354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1291/2609] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:15 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:15.623642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.633939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2043c-9ffb-429e-929c-f4deb432984b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.679546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:15.691215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.691882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.692167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.692445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.693126 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.695060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.716064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1319/2610] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:15 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:15.720811 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.731150 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe01b-2443-46f4-b740-ef2dcec7626e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.735741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.736336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.736621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.736897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.737573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.739147 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.748976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1292/2611] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:15.752711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.762372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6277b-7f37-48c9-bd39-7b46578a7078 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.767335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.767973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.768248 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.768593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.769350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.771371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773ca806-bc8d-44ca-81d8-f9fa248e1794 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.783331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1320/2612] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:15.787467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.797555 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44652-be6f-434c-85d8-9f357384d188 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:15.802279 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:15.802936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.803248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.803524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] PATH_INFO: `/v3/projects/317e9e95acf04c37bf5f0be34a3ccfa8/users/0094714df2b04b42b5d1c5a63da0ca08/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.819749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=317e9e95acf04c37bf5f0be34a3ccfa8, user_id=0094714df2b04b42b5d1c5a63da0ca08, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:15.822500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:15.836466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1293/2613] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:15 2022] PUT /identity/v3/projects/317e9e95acf04c37bf5f0be34a3ccfa8/users/0094714df2b04b42b5d1c5a63da0ca08/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:15.847378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:15.847600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:15.847874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:15.875999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] MFA Rules not processed for user `0094714df2b04b42b5d1c5a63da0ca08`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:15.877244 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:15.939909 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1321/2614] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:15.951051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:15.952772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:16.008226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8359be-6d77-4b95-8f02-145bfbed6c2f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:16.011828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:16.012422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:16.012704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:16.012989 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:16.020982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:16.022381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c19ed-acc9-48c5-a253-856a9b346c45 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:16.026817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:16.031724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1294/2615] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:25.936906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:40.919292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:40.981045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8bc5e-b6df-4a6d-9b27-02db97e57884 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:40.986396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:40.987078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:40.987411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:40.987778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] PATH_INFO: `/v3/users/0094714df2b04b42b5d1c5a63da0ca08` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:40.989121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0094714df2b04b42b5d1c5a63da0ca08)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:40.991366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.037691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0094714df2b04b42b5d1c5a63da0ca08'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.038390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0094714df2b04b42b5d1c5a63da0ca08'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.051016 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0094714df2b04b42b5d1c5a63da0ca08'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.058190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0094714df2b04b42b5d1c5a63da0ca08'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.065639 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0094714df2b04b42b5d1c5a63da0ca08'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.073649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1322/2616] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:57:40 2022] DELETE /identity/v3/users/0094714df2b04b42b5d1c5a63da0ca08 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:41.076239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.076766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.076875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.095763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:41.097026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.154741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1295/2617] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:57:41.165090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.166497 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.223189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5b691-55a5-4a92-a44d-3517d9285ce1 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.227392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.228071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.228439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.228721 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.238557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.240029 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144280f-a872-4bc2-b84e-3ddd8c85ce09 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.249551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1323/2618] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:41.513672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.515137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.527953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515cf90-c14c-4b5b-b1ce-82f725b1752c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.532789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.533529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.533828 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.534091 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] PATH_INFO: `/v3/projects/317e9e95acf04c37bf5f0be34a3ccfa8` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.535249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=317e9e95acf04c37bf5f0be34a3ccfa8)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.537170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.585069 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317e9e95acf04c37bf5f0be34a3ccfa8', 'request_id': 'req-cd6d1567-34d4-49de-9db0-d37aec66cb5f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.592291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317e9e95acf04c37bf5f0be34a3ccfa8', 'request_id': 'req-cd6d1567-34d4-49de-9db0-d37aec66cb5f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.599469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317e9e95acf04c37bf5f0be34a3ccfa8', 'request_id': 'req-cd6d1567-34d4-49de-9db0-d37aec66cb5f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.610653 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] The token cache is being invalidate because project 317e9e95acf04c37bf5f0be34a3ccfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:57:41.610997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.613503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cd6d1567-34d4-49de-9db0-d37aec66cb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:57:41.613503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1296/2619] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:57:41 2022] DELETE /identity/v3/projects/317e9e95acf04c37bf5f0be34a3ccfa8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:41.618197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.618583 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.618959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.643034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:41.644249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.661666 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:41.704582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1324/2620] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:41.713339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.713637 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.713918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.734120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:41.735330 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:41.743079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1297/2621] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:41.747196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.757340 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a1696-3522-41e3-ba07-cca9b7890744 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.761937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.762531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.762811 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.763088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.763724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:57:41.764034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.766172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.777652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1325/2622] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:57:41.782063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.791235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995eb70-ca44-4fad-a4a6-5db499bb1fcf None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.795847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.796440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.797248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.797520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.798137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.800010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.831542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1298/2623] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:57:41.832618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.842714 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4b85-f3c0-4d90-8c42-ece84b5cb2a2 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.905386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.905997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.906281 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.906560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.907204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.909175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.932099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1326/2624] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:57:41 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:57:41.936322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.947190 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c6ba-c432-4b15-9b7c-3f725a386b6f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.952271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.952894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.953235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.953512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.954138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.955708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.965486 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1299/2625] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:57:41.969958 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:41.979980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df34b-a2bc-4ccd-a72e-e76cd7eefefb admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:41.985277 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:41.985873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:41.986156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:41.986431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:41.987057 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:41.988661 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8014dce-f94d-4f2f-bd8a-cba5114c93a5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:41.999066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1327/2626] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:57:42.003016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:42.012764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5fd38-d045-42a1-b0e2-e4438abc234f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:42.017380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:42.018028 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.018308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.018605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] PATH_INFO: `/v3/projects/5906bfeb2bd24a289ce3232c44ffa95a/users/306a7feefe924fcdb869f8cc34f04c4d/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.033373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5906bfeb2bd24a289ce3232c44ffa95a, user_id=306a7feefe924fcdb869f8cc34f04c4d, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:42.036008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:42.049336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1300/2627] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:57:42 2022] PUT /identity/v3/projects/5906bfeb2bd24a289ce3232c44ffa95a/users/306a7feefe924fcdb869f8cc34f04c4d/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:57:42.059771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9ae4-2c56-4e3e-9122-ab97c6fd1514 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.060094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9ae4-2c56-4e3e-9122-ab97c6fd1514 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.060388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9ae4-2c56-4e3e-9122-ab97c6fd1514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.091269 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-287b9ae4-2c56-4e3e-9122-ab97c6fd1514 None None] MFA Rules not processed for user `306a7feefe924fcdb869f8cc34f04c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:42.092988 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-287b9ae4-2c56-4e3e-9122-ab97c6fd1514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:42.159464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1328/2628] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:42.171117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:42.173633 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:42.195572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:42.231764 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970e51-5652-4f49-87b1-4e03a9f012a5 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:42.236047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:42.236650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:42.236932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:42.237236 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:42.245442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:42.248635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:42.259615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1301/2629] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:57:44.667556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb9f3-6442-489e-ab0e-0307095a6c48 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:44.668121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb9f3-6442-489e-ab0e-0307095a6c48 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:44.668121 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb9f3-6442-489e-ab0e-0307095a6c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:44.688794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-81bbb9f3-6442-489e-ab0e-0307095a6c48 None None] MFA Rules not processed for user `306a7feefe924fcdb869f8cc34f04c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:44.690031 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-81bbb9f3-6442-489e-ab0e-0307095a6c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:44.696818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1329/2630] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:47.908868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:47.910440 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:47.987597 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bded7b-062d-40c9-96cf-4dc60bd9f22b service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:47.993731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:47.994466 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:47.994765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:47.995080 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:47.997176 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:48.011393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:48.013141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:48.024570 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1302/2631] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:54.682622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:58.019954 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=51 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:04.167668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bd2b-0cca-4066-9bc5-fc65005b3460 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:04.168092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bd2b-0cca-4066-9bc5-fc65005b3460 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:04.168406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bd2b-0cca-4066-9bc5-fc65005b3460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:04.218592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-2ea8bd2b-0cca-4066-9bc5-fc65005b3460 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:04.219790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2ea8bd2b-0cca-4066-9bc5-fc65005b3460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:04.282412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1330/2632] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:04.328883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:04.330319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:04.419119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae600a1d-50e5-4910-a623-32d7170e4f8e service nova] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:04.424467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:04.425096 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:04.425374 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:04.425644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:04.433644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:04.435187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:04.443491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1303/2633] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:04.943860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e37a1-a610-4d25-9a84-63ebebc0fe53 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:04.944146 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e37a1-a610-4d25-9a84-63ebebc0fe53 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:04.944421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e37a1-a610-4d25-9a84-63ebebc0fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:04.963950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6e2e37a1-a610-4d25-9a84-63ebebc0fe53 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:04.965167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6e2e37a1-a610-4d25-9a84-63ebebc0fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:04.972664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1331/2634] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 09:58:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:04.984625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:04.995244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889ef39-5e07-4f4b-9124-6465f7f8df26 service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:05.000250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:05.000839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:05.001165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:05.001438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:05.009547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:05.011103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:05.019533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1304/2635] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 09:58:05.235303 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdff5-b095-4e3b-8139-c30df3a841fd None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:05.235771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdff5-b095-4e3b-8139-c30df3a841fd None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:05.235880 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcdff5-b095-4e3b-8139-c30df3a841fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:05.254903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f3dcdff5-b095-4e3b-8139-c30df3a841fd None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:05.256086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f3dcdff5-b095-4e3b-8139-c30df3a841fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:05.263378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1332/2636] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:05.275089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:05.286004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9475ce-c6d4-427d-b727-533587dbbe5e service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:05.290875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:05.291459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:05.291739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:05.292012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:05.300480 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:05.302063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:05.310191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1305/2637] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:05.634079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:05.634370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:05.634644 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:05.654215 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:05.655977 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:05.663034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1333/2638] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 09:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:05.675047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:05.685809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd9b6-47da-427b-afcc-c5bf6ce29627 service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:05.690920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:05.691513 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:05.691796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:05.692071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:05.700284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:05.702311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:05.710166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1306/2639] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:06.061453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:06.062899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:06.079447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9a0d-251c-45f4-99e3-654444100f53 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:06.085812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:06.086331 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:06.086612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:06.086887 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:06.088624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:06.169687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:06.171220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c60d9a-653f-4e9b-b3eb-b4392a261158 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:06.190132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1334/2640] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:08.879409 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:08.945520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333839-0da9-405d-aa1a-909adfe00708 service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:08.952772 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:08.953442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:08.953726 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:08.954004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] PATH_INFO: `/v3/users/306a7feefe924fcdb869f8cc34f04c4d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:08.955131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306a7feefe924fcdb869f8cc34f04c4d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:08.957054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:08.993098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:09.006450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306a7feefe924fcdb869f8cc34f04c4d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.015369 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306a7feefe924fcdb869f8cc34f04c4d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.027806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306a7feefe924fcdb869f8cc34f04c4d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.035191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306a7feefe924fcdb869f8cc34f04c4d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.035865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306a7feefe924fcdb869f8cc34f04c4d'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.042135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1307/2641] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:08 2022] DELETE /identity/v3/users/306a7feefe924fcdb869f8cc34f04c4d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:09.047073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.047537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.047636 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.067209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:09.068388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:09.086262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:09.133579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1335/2642] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:09.144127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.145559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:09.204304 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c572d-bafe-41be-bf47-6a6f09c4f033 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.208290 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.208893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.209206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.209483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.219164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.220569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69510-b5f6-47ec-a8fd-77db255fe293 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.229829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1308/2643] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:09.401498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.402959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:09.415341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a1180-ccdf-46df-899f-8a527cf74c2c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.420019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.420684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.420984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.421273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] PATH_INFO: `/v3/projects/5906bfeb2bd24a289ce3232c44ffa95a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.422469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5906bfeb2bd24a289ce3232c44ffa95a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.424382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.474760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5906bfeb2bd24a289ce3232c44ffa95a', 'request_id': 'req-fc3b700d-4a04-4fc1-b992-7e57d059ae93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.480039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5906bfeb2bd24a289ce3232c44ffa95a', 'request_id': 'req-fc3b700d-4a04-4fc1-b992-7e57d059ae93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.486259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5906bfeb2bd24a289ce3232c44ffa95a', 'request_id': 'req-fc3b700d-4a04-4fc1-b992-7e57d059ae93'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.498122 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] The token cache is being invalidate because project 5906bfeb2bd24a289ce3232c44ffa95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:09.498411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.499034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-fc3b700d-4a04-4fc1-b992-7e57d059ae93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:09.500538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1336/2644] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:09 2022] DELETE /identity/v3/projects/5906bfeb2bd24a289ce3232c44ffa95a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:09.505512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.505804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.506110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.525707 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:09.526964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:09.589743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1309/2645] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:09.598138 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.598435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.598713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.618315 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:09.619519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:09.627469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1337/2646] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:09.631579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.641658 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f5645-0091-4369-9c0c-6215421e3ad4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.646178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.646798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.647075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.647345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.647976 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:09.648273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.649867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.661565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1310/2647] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:09.665826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.675613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9411eb-6b75-4630-af63-84dc31c92c6b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.681106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.681711 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.681993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.682261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.682895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.684829 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.713683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1338/2648] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:09.718502 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.728759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c1ab2-37e9-472d-a24d-74a19e42da41 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.767520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:09.787377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.787982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.788270 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.788543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.789234 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.791113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.812009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1311/2649] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:09.816223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.826426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b533f-0550-478c-85b2-8856cc357c1f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.830978 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.831571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.831851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.832123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.832759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.834379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.844547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1339/2650] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:09.848520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.859049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccb55e-d60c-4800-b1bd-d823bfaa1851 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.863677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.864297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.864581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.864857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.865525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.867089 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682511e2-f824-4437-8f94-8367dfb680df admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.876880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1312/2651] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:09.880697 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:09.890375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9203b-bb10-41b5-a153-fecddf662af9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:09.894833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:09.895468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.895743 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.896013 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] PATH_INFO: `/v3/projects/fa802df85bc948db9531867b47c45079/users/ede676b2a0e146d3ab23edd7e3583f15/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:09.910510 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa802df85bc948db9531867b47c45079, user_id=ede676b2a0e146d3ab23edd7e3583f15, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:09.913161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:09.926961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1340/2652] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:09 2022] PUT /identity/v3/projects/fa802df85bc948db9531867b47c45079/users/ede676b2a0e146d3ab23edd7e3583f15/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:09.992285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:09.992836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:09.992836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:10.021620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] MFA Rules not processed for user `ede676b2a0e146d3ab23edd7e3583f15`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:10.022835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:10.087387 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1313/2653] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:10.098438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:10.100216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:10.145159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:10.158719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3bd9-d912-4211-a796-19b45157a049 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:10.162311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:10.162908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:10.163192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:10.163460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:10.173050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:10.174530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:10.183521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1341/2654] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:11.345398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:11.346784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:11.359520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527b60c-b73c-44dc-b57d-e248cb94b2ac None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:11.363060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:11.363690 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:11.363972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:11.364247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:11.365960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:11.445520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:11.446948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:11.465384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1314/2655] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:20.179592 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:21.463184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:34.530566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:34.532026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:34.588609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f865a8-4edf-4eeb-8575-77217b7e1c55 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:34.592658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:34.593298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:34.593579 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:34.593874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:34.595672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:34.662881 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:34.664355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:34.675071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1342/2656] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:38.334978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.338939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.411137 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.412577 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.414874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ba59-45e8-47cd-8633-7c15b410fb61 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.422622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.423282 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.423884 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.424160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.426005 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.426120 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141944a-5246-4764-a590-984fb59afbb3 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.431129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.431900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.432191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.432467 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] PATH_INFO: `/v3/users/468522570ba7419c87da6d05fc0d1ff1` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.433673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468522570ba7419c87da6d05fc0d1ff1)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.435663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.439038 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.440727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.451218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1315/2657] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:38.489279 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468522570ba7419c87da6d05fc0d1ff1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.489946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468522570ba7419c87da6d05fc0d1ff1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.503622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468522570ba7419c87da6d05fc0d1ff1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.510496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468522570ba7419c87da6d05fc0d1ff1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.518468 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f14f467c-a5a7-454a-b2b7-c31ca85b58b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468522570ba7419c87da6d05fc0d1ff1'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.524560 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1343/2658] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:38 2022] DELETE /identity/v3/users/468522570ba7419c87da6d05fc0d1ff1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:38.697163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.715936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c469-ce02-4351-b16f-d8df8a110e9e service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.774967 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.775605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.775908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.776163 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] PATH_INFO: `/v3/projects/d16af15e86174ab1b9344d35ffe0844d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.777358 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16af15e86174ab1b9344d35ffe0844d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.780016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.825285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16af15e86174ab1b9344d35ffe0844d', 'request_id': 'req-f6310a35-b3f6-4717-9283-6365accb573b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.832543 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16af15e86174ab1b9344d35ffe0844d', 'request_id': 'req-f6310a35-b3f6-4717-9283-6365accb573b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.838694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16af15e86174ab1b9344d35ffe0844d', 'request_id': 'req-f6310a35-b3f6-4717-9283-6365accb573b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.850436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] The token cache is being invalidate because project d16af15e86174ab1b9344d35ffe0844d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:38.850812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.851379 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:38.852898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1316/2659] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:38 2022] DELETE /identity/v3/projects/d16af15e86174ab1b9344d35ffe0844d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:38.858035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d23b4e-d872-41ab-92ca-86f1656bc0b4 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.858321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d23b4e-d872-41ab-92ca-86f1656bc0b4 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.858597 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d23b4e-d872-41ab-92ca-86f1656bc0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.872978 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.874489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.878821 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-43d23b4e-d872-41ab-92ca-86f1656bc0b4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:38.879985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-43d23b4e-d872-41ab-92ca-86f1656bc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.896969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:38.898049 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:38.942081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6310a35-b3f6-4717-9283-6365accb573b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:38.945074 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1344/2660] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:38.946682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:38.947322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.947605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.947877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] PATH_INFO: `/v3/users/ede676b2a0e146d3ab23edd7e3583f15` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.949031 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede676b2a0e146d3ab23edd7e3583f15)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:38.951438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:38.953240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:38.953530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:38.953813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:38.974077 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:38.975384 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:38.983866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1345/2661] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:38.988011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:38.997477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8efa44-0a55-4658-990e-87b0bb7f25e8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.000220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede676b2a0e146d3ab23edd7e3583f15'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.002073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.002662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.002951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.003226 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.003869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:39.004180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.005816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.008114 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede676b2a0e146d3ab23edd7e3583f15'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.018211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1346/2662] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:58:39.022376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.027046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede676b2a0e146d3ab23edd7e3583f15'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.032442 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb572b9e-a532-4f30-838a-1504d4056c02 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.034212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede676b2a0e146d3ab23edd7e3583f15'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.034907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-24265cb1-7c06-4b4f-8d16-5b17b7f404b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede676b2a0e146d3ab23edd7e3583f15'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.037060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.037663 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.037945 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.038216 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.038819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.040678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.041300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1317/2663] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:38 2022] DELETE /identity/v3/users/ede676b2a0e146d3ab23edd7e3583f15 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:39.045612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.045901 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.046180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.067583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:39.073885 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.074040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1347/2664] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:39.074040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.075748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.154813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7732c7-e333-4356-88fd-e37f1b5858bb admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.156559 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1318/2665] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:39.161748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.162443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.162802 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.163140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.163841 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.166052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.167013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.168518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.196339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1348/2666] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:39.200219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.211300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c2e4c-4a54-411d-a1b4-941259fff90c admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.217728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.218339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.218615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.218885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.220465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.222313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.225284 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:39.234851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1349/2667] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:39.238506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.246446 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afd0df-e4a6-4d42-b306-fb4287904243 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.250474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.251092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.251399 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.251678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.253464 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e704bb-fe97-4d29-9b98-979dde945411 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.260116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.260819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.262207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.262207 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.262952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.263210 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.264655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.264942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c5c4a-21bd-44b2-ab50-57b4e6515b6d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.275803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1319/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:39.278363 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1350/2669] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:39.282524 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.293199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a603a3-f261-40de-9331-165a75ccbb27 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.299461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.300373 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.300662 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.300945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] PATH_INFO: `/v3/projects/b25d6c9e17474409a17d93c188dc3f20/users/d3119d129aec473495bbae2e54cc0177/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.318738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25d6c9e17474409a17d93c188dc3f20, user_id=d3119d129aec473495bbae2e54cc0177, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.321488 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.345075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1320/2670] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:39 2022] PUT /identity/v3/projects/b25d6c9e17474409a17d93c188dc3f20/users/d3119d129aec473495bbae2e54cc0177/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:39.351156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.351567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.351900 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.388668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] MFA Rules not processed for user `d3119d129aec473495bbae2e54cc0177`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:39.390154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.433624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:39.467336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1351/2671] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:39.477892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.479612 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.557595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49670fd0-8402-4eaf-97e7-8b4e28032873 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.564012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.564657 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.565065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.565341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.573873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.575875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa22350-5422-45a6-951d-862731b04c6f service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.581672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.583498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.585703 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1321/2672] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:39.673968 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6fdb0-25a2-42bb-a088-a597afa8ad13 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.678876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.679523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.679804 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.680080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] PATH_INFO: `/v3/projects/fa802df85bc948db9531867b47c45079` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.681235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa802df85bc948db9531867b47c45079)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.683211 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.735552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa802df85bc948db9531867b47c45079', 'request_id': 'req-08c11254-202c-41ee-a6c2-18fe78f6019c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.742358 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa802df85bc948db9531867b47c45079', 'request_id': 'req-08c11254-202c-41ee-a6c2-18fe78f6019c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.749866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa802df85bc948db9531867b47c45079', 'request_id': 'req-08c11254-202c-41ee-a6c2-18fe78f6019c'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.764858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] The token cache is being invalidate because project fa802df85bc948db9531867b47c45079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:39.765199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.765893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-08c11254-202c-41ee-a6c2-18fe78f6019c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:39.767725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1352/2673] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:39 2022] DELETE /identity/v3/projects/fa802df85bc948db9531867b47c45079 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:39.773175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.773487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.773812 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.795685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:39.796919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.876143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1322/2674] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:39.878050 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.878414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.878765 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.904338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:39.905764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:39.922104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1353/2675] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:39.926332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.937444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29658-df00-4ae8-9727-559210fd2d56 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.942322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.942939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.943223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.943501 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.944146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:39.944455 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.946172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:39.959421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1323/2676] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:39.963738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.974129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cc967-150d-4542-a361-ae9cd9964129 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:39.979368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:39.979991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:39.980252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:39.980525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:39.981395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:39.983345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.010954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1354/2677] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:39 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:40.015864 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.025783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18885e-5391-42d9-a3d9-40950abc7be8 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.054442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:40.083795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.084406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.084687 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.085051 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.085708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.087624 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.109627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1324/2678] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:40 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:40.113725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.124654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c77469-4f14-4a4c-be24-d6205f60e579 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.129364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.129959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.130242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.130519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.131157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.132756 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.142552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1355/2679] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:40.146228 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.156316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bbace-8bd0-4b8e-848c-3c5bfc03249e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.160939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.161553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.161836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.162109 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.162741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.164332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e7a55-48f6-415f-bfcd-166cb0b1ee2a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.176103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1325/2680] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:40.180672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.191309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3bc17-1747-46df-8feb-ce4fed76ad00 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.195844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.196530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.196810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.197110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] PATH_INFO: `/v3/projects/df7a0ee4a83f48d78f1c941172cacdb9/users/633f1828cf87496fbc0dad682bf9986b/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.211320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7a0ee4a83f48d78f1c941172cacdb9, user_id=633f1828cf87496fbc0dad682bf9986b, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.213995 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.226164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1356/2681] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:40 2022] PUT /identity/v3/projects/df7a0ee4a83f48d78f1c941172cacdb9/users/633f1828cf87496fbc0dad682bf9986b/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:40.231715 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.232002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.232274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.251492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:40.252682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:40.310778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1326/2682] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:40.320321 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.322051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:40.328994 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:40.388926 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff40150-7e03-4db3-ace9-2877133d8f9f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.392586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.393249 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.393528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.393849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.402969 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.404387 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a540133-703e-4b5d-bfef-16a0bb7b2793 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.420060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1357/2683] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:41.726205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.728024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:41.740857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067cdc0-f34b-40f0-9313-9e5be8c137b3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.744946 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.745573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.745849 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.746119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.748883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:41.763454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.764741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.777797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1327/2684] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:45.142778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.143693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.144140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.179713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] MFA Rules not processed for user `633f1828cf87496fbc0dad682bf9986b`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:45.180949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:45.263537 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1358/2685] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:45.264495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.280541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a193c-26fb-444b-bdb3-aad01e425e67 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.288003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.288625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.288907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.289287 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.289936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.291868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.336235 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1328/2686] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:45 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:45.336381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.349136 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b0dc1-5e54-42f1-98c9-a0596ed4399c None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.446485 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.447368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.447824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.448245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.449591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.452815 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.487619 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1359/2687] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:45 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:45.491674 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.504584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8685777-26e4-440d-b35d-8907e65f0904 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.516128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.517104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.517538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.517981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.518791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.520798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.542413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.544103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1329/2688] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:45.563155 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e54168-38da-40b5-a059-4e654a8876ad admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.567983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.568643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.568924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.569228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] PATH_INFO: `/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3/users/94127ba08f9a41deb75c1047359aedf0/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.610313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce30a91271b4c2aaf47560f5d2e65b3, user_id=94127ba08f9a41deb75c1047359aedf0, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.614234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.636259 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1360/2689] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:45 2022] PUT /identity/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3/users/94127ba08f9a41deb75c1047359aedf0/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:45.637077 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.665792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63da7f-dd04-4eaa-981d-fa2c1e816574 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.738723 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.739470 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.739988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.740425 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.741311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.742916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.749678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:45.764214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.766126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1330/2690] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:45.770779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:45.778431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de747-9381-40bc-85aa-44d30b580cb4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.785432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.787163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.787540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.788691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] PATH_INFO: `/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3/users/94127ba08f9a41deb75c1047359aedf0/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.790931 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce30a91271b4c2aaf47560f5d2e65b3, user_id=94127ba08f9a41deb75c1047359aedf0, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.794333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.811803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1361/2691] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:45 2022] PUT /identity/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3/users/94127ba08f9a41deb75c1047359aedf0/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:45.861847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.863873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:45.881848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabda5f9-8751-4867-ab4d-f1efb1bdd82a admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.949882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.950485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.950769 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.951046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] PATH_INFO: `/v3/users/0fca332203b9473fa8b07286bc44300f` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.952216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fca332203b9473fa8b07286bc44300f)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.954357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:46.006095 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fca332203b9473fa8b07286bc44300f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.015936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fca332203b9473fa8b07286bc44300f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.029180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fca332203b9473fa8b07286bc44300f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.036263 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fca332203b9473fa8b07286bc44300f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.037022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-935199a6-1277-4b34-860c-8156490d5421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fca332203b9473fa8b07286bc44300f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.043663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1331/2692] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:45 2022] DELETE /identity/v3/users/0fca332203b9473fa8b07286bc44300f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:46.364587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:46.367600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:46.385079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db4ddc-ea73-44ae-8b3c-672f8985fce9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:46.475924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:46.477099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.477447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.477792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] PATH_INFO: `/v3/projects/1e3cef1ca6964f3dbae8e170ff530811` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.479299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3cef1ca6964f3dbae8e170ff530811)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:46.481963 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:46.590369 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3cef1ca6964f3dbae8e170ff530811', 'request_id': 'req-6065f79d-54fc-4719-bd8b-95c6c1e441ef'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.592840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3cef1ca6964f3dbae8e170ff530811', 'request_id': 'req-6065f79d-54fc-4719-bd8b-95c6c1e441ef'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.601118 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3cef1ca6964f3dbae8e170ff530811', 'request_id': 'req-6065f79d-54fc-4719-bd8b-95c6c1e441ef'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.621445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] The token cache is being invalidate because project 1e3cef1ca6964f3dbae8e170ff530811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:46.622024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.622698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6065f79d-54fc-4719-bd8b-95c6c1e441ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:46.624534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1362/2693] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:46 2022] DELETE /identity/v3/projects/1e3cef1ca6964f3dbae8e170ff530811 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:46.630357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.630892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.630892 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.654856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:46.656074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:46.764201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1332/2694] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:46.771853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.772382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.772830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.796278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:46.797519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:46.807430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1363/2695] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:46.812011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:46.822453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed5bbf-6a7e-4a7a-bd5e-ffde740a2a13 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:46.827317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:46.827927 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.828255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.828467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.829153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:46.829449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:46.831034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:46.843692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1333/2696] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:46.848423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:46.859860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd64b4-1bec-4395-a1f2-1f2692affd70 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:46.866386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:46.867149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.867531 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.868142 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.869380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:46.871899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:46.909849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1364/2697] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:46 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:46.913893 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:46.924685 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d4e4-a0b5-4446-8b5c-307beff19e5f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:46.928607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:46.989042 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:46.989734 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:46.990084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:46.990413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:46.991099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:46.993075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.017607 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1334/2698] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:46 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:47.021606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:47.032890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584870e-02c4-464f-a6ef-e1ce0b772937 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:47.037680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:47.038700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.038700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.039116 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.039839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:47.041710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.054860 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1365/2699] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:47.058880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:47.068815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aad0a-bd10-4399-a39a-8fc21ccc0bad admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:47.082086 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:47.082615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.083424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.083819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.084584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:47.086713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9582ad-87d0-4b71-904e-fc8853248f04 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.098877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1335/2700] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:47.102939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:47.113443 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08f986-1e42-4e35-8c10-30c21f1968bf admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:47.118847 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60450684-4392-434b-b801-015cdb524216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:47.119640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60450684-4392-434b-b801-015cdb524216 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.120110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60450684-4392-434b-b801-015cdb524216 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.120311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60450684-4392-434b-b801-015cdb524216 admin admin] PATH_INFO: `/v3/projects/b5cf1d64c45243ddad8248dac63ac71a/users/74d4e98641b84c5cb1b4c61d753e5054/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.138017 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60450684-4392-434b-b801-015cdb524216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5cf1d64c45243ddad8248dac63ac71a, user_id=74d4e98641b84c5cb1b4c61d753e5054, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:47.140878 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60450684-4392-434b-b801-015cdb524216 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.154951 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1366/2701] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:47 2022] PUT /identity/v3/projects/b5cf1d64c45243ddad8248dac63ac71a/users/74d4e98641b84c5cb1b4c61d753e5054/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:47.169323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.169801 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.170145 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.203858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] MFA Rules not processed for user `74d4e98641b84c5cb1b4c61d753e5054`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:47.205124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:47.291527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1336/2702] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:47.299624 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60450684-4392-434b-b801-015cdb524216 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:47.301957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-60450684-4392-434b-b801-015cdb524216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:47.325289 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:47.376286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60450684-4392-434b-b801-015cdb524216 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:47.381090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:47.381906 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.382182 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.382627 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.392090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:47.393794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e485e-fb63-4791-9005-5385fb343261 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.404422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1367/2703] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:50.587652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:50.590247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:50.608171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:50.608710 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:50.610774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:50.643453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] MFA Rules not processed for user `94127ba08f9a41deb75c1047359aedf0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:50.644919 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:50.738563 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f9335-a7d5-4e82-90e9-393c42666404 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:50.745404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:50.746081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:50.746361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:50.746635 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] PATH_INFO: `/v3/users/d3119d129aec473495bbae2e54cc0177` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:50.747765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3119d129aec473495bbae2e54cc0177)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:50.749713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:50.762645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1368/2704] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:50.771245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:50.775471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:50.777798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:50.825083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3119d129aec473495bbae2e54cc0177'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.837613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3119d129aec473495bbae2e54cc0177'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.858161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3119d129aec473495bbae2e54cc0177'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.871170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3119d129aec473495bbae2e54cc0177'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.872822 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-9b752ec8-c8ce-4dc4-98ac-50f60ccdbe95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3119d129aec473495bbae2e54cc0177'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.885012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1337/2705] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:50 2022] DELETE /identity/v3/users/d3119d129aec473495bbae2e54cc0177 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:50.889151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:50.889447 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:50.889728 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:50.917975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:50.919415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:50.923551 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdb450-961b-4575-accd-e069702c2de2 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:50.970520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=100 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:50.984422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b91846cfa8f40109e3d3d10dec3adc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:50.985172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:50.985487 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:50.985897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:50.988124 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.009572 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1338/2706] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:51.020166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.023268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.078689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.080156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75862054-238c-4f1e-9fa1-5cd2d06ca7fc service glance] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.093165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ca907-f2c2-454f-890b-ddfd0dff2bb4 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.094655 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1369/2707] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:51.097338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.097958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.098241 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.098519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.108439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.109918 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.120036 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1339/2708] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:51.246260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.246471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.246744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.269282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:51.270436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.336539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1370/2709] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:51.350212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.351713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.407962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:51.451446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.452873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671766e1-b645-4600-be9f-0ee5e3f99dee service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.458208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.458838 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.459115 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.459398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.460163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.479027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.479305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703ca7c-119d-4883-84e4-128647d4ad7c service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.483640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e983-3027-4032-a27d-533f32a6e20b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.490438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.491092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.491366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.491634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] PATH_INFO: `/v3/projects/b25d6c9e17474409a17d93c188dc3f20` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.492854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25d6c9e17474409a17d93c188dc3f20)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.495033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.506818 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1340/2710] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:51.551795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25d6c9e17474409a17d93c188dc3f20', 'request_id': 'req-0720ae09-7c94-430c-8c58-8741f53937b9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.569607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25d6c9e17474409a17d93c188dc3f20', 'request_id': 'req-0720ae09-7c94-430c-8c58-8741f53937b9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.580799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25d6c9e17474409a17d93c188dc3f20', 'request_id': 'req-0720ae09-7c94-430c-8c58-8741f53937b9'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.600480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] The token cache is being invalidate because project b25d6c9e17474409a17d93c188dc3f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:51.600480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.601330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-0720ae09-7c94-430c-8c58-8741f53937b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.606893 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1371/2711] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:51 2022] DELETE /identity/v3/projects/b25d6c9e17474409a17d93c188dc3f20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:51.611936 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.612419 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.612614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.634445 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:51.635935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.709044 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1341/2712] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:51.716795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.717350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.717469 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.742034 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:51.743230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.752460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1372/2713] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:51.756615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.768512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2f16-b8b2-407f-adf4-83474f2e4163 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.773718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.774331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.774606 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.774999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.775652 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:51.775960 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.778117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.791317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1342/2714] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:51.795885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.811006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec9798-c689-451d-ab05-a993e9e917c7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.818413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.819223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.819674 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.819949 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.820599 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.822590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.844454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:51.864458 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1373/2715] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:51.868829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.882659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c2a42-535f-402c-8506-8fbddb36552f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.960016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.960639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.960917 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.961240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.961872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.964331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.990679 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1343/2716] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:51.994924 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.009545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87a9c0-37f3-4a6d-8891-46587607b96f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.015421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.016100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.016451 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.016797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.019526 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.021262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.033748 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1374/2717] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:52.039301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.049642 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a43d9f-0b72-4d19-89cb-a3561cd6273c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.054693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.055301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.055583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.055858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.056493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.058623 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07649fa-19d1-468e-9d14-5596047ddebd admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.077988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1344/2718] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:52.082623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.095129 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde62c2-8c45-4f8c-af22-ac599fc48636 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.102656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.103336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.103618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.103894 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] PATH_INFO: `/v3/projects/f554d6e8617642eb8a5665973f29c570/users/3e33d89eeef74a0fbcb0225ce146c4d3/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.124258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f554d6e8617642eb8a5665973f29c570, user_id=3e33d89eeef74a0fbcb0225ce146c4d3, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.127035 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.141193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1375/2719] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:52 2022] PUT /identity/v3/projects/f554d6e8617642eb8a5665973f29c570/users/3e33d89eeef74a0fbcb0225ce146c4d3/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:52.154880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.155165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.155441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.217583 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] MFA Rules not processed for user `3e33d89eeef74a0fbcb0225ce146c4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:52.219385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.269064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:52.350477 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1345/2720] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:52.361685 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.363989 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.507339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590c1f7-5723-4f14-ab5b-f84d28dab4cb admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.513928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.514984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.515264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.515547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.528839 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.537543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac72931-dcad-42c6-8d2a-50c9e24c01dd service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.555404 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1376/2721] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:58.749991 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:58.751553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:58.836349 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77927bb4-4677-4387-9555-3068f4101bbe None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:58.842390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:58.844975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:58.844975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:58.845225 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] PATH_INFO: `/v3/users/74d4e98641b84c5cb1b4c61d753e5054` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:58.846546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d4e98641b84c5cb1b4c61d753e5054)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:58.848602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:58.905260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d4e98641b84c5cb1b4c61d753e5054'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:58.913518 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d4e98641b84c5cb1b4c61d753e5054'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:58.929920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d4e98641b84c5cb1b4c61d753e5054'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:58.937237 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d4e98641b84c5cb1b4c61d753e5054'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:58.938073 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d4e98641b84c5cb1b4c61d753e5054'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:58.946329 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1346/2722] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:58 2022] DELETE /identity/v3/users/74d4e98641b84c5cb1b4c61d753e5054 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:58.951461 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15462d57-0afe-411f-8988-81952efd719a None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:58.951982 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15462d57-0afe-411f-8988-81952efd719a None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:58.952083 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15462d57-0afe-411f-8988-81952efd719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:58.972707 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-15462d57-0afe-411f-8988-81952efd719a None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:58.973972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-15462d57-0afe-411f-8988-81952efd719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:59.052745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1377/2723] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:59.065288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:59.066869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:59.124406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:59.144325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64487ed-e7f1-4eb0-9790-c7c299c1f241 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:59.148238 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:59.148863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.149178 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.149459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.160177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:59.161650 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccd6b1-5137-4678-8a47-80607454577b service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:59.177164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1347/2724] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:59.475197 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15462d57-0afe-411f-8988-81952efd719a None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:59.477964 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-15462d57-0afe-411f-8988-81952efd719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:59.509366 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15462d57-0afe-411f-8988-81952efd719a None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:59.516682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:59.517456 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.517810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.518105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] PATH_INFO: `/v3/projects/b5cf1d64c45243ddad8248dac63ac71a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.519819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5cf1d64c45243ddad8248dac63ac71a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:59.522245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:59.594322 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5cf1d64c45243ddad8248dac63ac71a', 'request_id': 'req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:59.601706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:59.603796 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5cf1d64c45243ddad8248dac63ac71a', 'request_id': 'req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:59.617816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5cf1d64c45243ddad8248dac63ac71a', 'request_id': 'req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:59.636659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] The token cache is being invalidate because project b5cf1d64c45243ddad8248dac63ac71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:59.637023 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:59.637656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-7a45ce7b-4c9b-48c2-87d8-f176df43d92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:59.639540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1378/2725] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:59 2022] DELETE /identity/v3/projects/b5cf1d64c45243ddad8248dac63ac71a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:59.644148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.644663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.645045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.669935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:59.670987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:59.739595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1348/2726] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:59.749012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.749567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.750042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.774565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:59.776060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:59.787775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1379/2727] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:59.791894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:59.802541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ebc4a-7fb5-4369-acab-acf36d596e15 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:59.807576 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:59.808835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.809156 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.809431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.810084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:59.810390 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:59.811982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:59.826025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1349/2728] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:59.830330 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:59.840076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e603e-91d9-495e-9c99-44ac6677bca6 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:59.845824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:59.846454 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.846733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.847010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.847651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:59.849630 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:59.883947 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1380/2729] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:59 2022] POST /identity/v3/projects => generated 359 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:59.888647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:59.899907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2ff36-8585-4c9e-9a15-1ea85543f53d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:59.970424 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:59.971060 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:59.971337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:59.971610 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:59.972252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:59.974258 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:59.998974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1350/2730] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:59 2022] POST /identity/v3/users => generated 280 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:00.006431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.017733 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479c2ed-128e-4aa5-95e6-e8ad6ef00855 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.022729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.023440 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.023780 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.024063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.024725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.026976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.041725 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1381/2731] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:00.043868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.057859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0d51-4694-4e40-a1f1-c6883c5e4e11 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.063110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.063759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.064046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.064326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.068026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.068026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf74f7-b269-4d8c-9cb2-8e120c455b76 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.080315 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1351/2732] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:00.088061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.111441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504988-bb3e-41df-80a1-f1090c56d210 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.116395 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.117285 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.117567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.117843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] PATH_INFO: `/v3/projects/175e05a0f4864d4f86ffaa8bd74c0471/users/49945dfc05354956a1a29cd76ca59775/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.143411 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e05a0f4864d4f86ffaa8bd74c0471, user_id=49945dfc05354956a1a29cd76ca59775, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.146398 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.160557 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1382/2733] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:00 2022] PUT /identity/v3/projects/175e05a0f4864d4f86ffaa8bd74c0471/users/49945dfc05354956a1a29cd76ca59775/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:00.171460 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.171750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.172121 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.203383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] MFA Rules not processed for user `49945dfc05354956a1a29cd76ca59775`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:00.204750 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:00.251420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:00.283368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1352/2734] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:00.320252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.322183 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:00.403313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b594e-b092-4484-a4be-4ee74ff80521 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.407364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.408095 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.408392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.408660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.419335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.421148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c09c28-4ea7-4d31-a0c9-bfdf94fbf753 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.440633 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1383/2735] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:00.447789 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.528035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04811082-a50e-45de-8cf5-56d65f9a723e None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.534844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.535721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.536076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.536422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] PATH_INFO: `/v3/users/49945dfc05354956a1a29cd76ca59775` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.537701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49945dfc05354956a1a29cd76ca59775)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.540568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.620062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49945dfc05354956a1a29cd76ca59775'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:00.637035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49945dfc05354956a1a29cd76ca59775'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:00.665556 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49945dfc05354956a1a29cd76ca59775'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:00.673682 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49945dfc05354956a1a29cd76ca59775'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:00.674580 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49945dfc05354956a1a29cd76ca59775'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:00.682016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1353/2736] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:00 2022] DELETE /identity/v3/users/49945dfc05354956a1a29cd76ca59775 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:00.690582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28518c-9127-4b2d-8922-0bb0f8250a36 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.690944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28518c-9127-4b2d-8922-0bb0f8250a36 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.691294 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28518c-9127-4b2d-8922-0bb0f8250a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.713741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-af28518c-9127-4b2d-8922-0bb0f8250a36 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:00.715629 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-af28518c-9127-4b2d-8922-0bb0f8250a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:00.784337 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1384/2737] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:00.794694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:00.796208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:00.842728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758429-0682-4d50-b2e6-a705fb495740 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.843064 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758429-0682-4d50-b2e6-a705fb495740 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.843286 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758429-0682-4d50-b2e6-a705fb495740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.863759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb6eb0-66fc-4dd1-9ac3-858cb766a578 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:00.868725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:00.869230 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f9758429-0682-4d50-b2e6-a705fb495740 None None] MFA Rules not processed for user `633f1828cf87496fbc0dad682bf9986b`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:00.869381 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:00.869659 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:00.869995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:00.871260 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f9758429-0682-4d50-b2e6-a705fb495740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:00.880224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:00.882141 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:00.892658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:00.898719 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1354/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:00.940713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1385/2739] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:01.231525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.235035 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.255016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:01.271286 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee249101-3bad-4c88-842f-e8864d03421a service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.276649 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.277439 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.277788 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.278131 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] PATH_INFO: `/v3/projects/175e05a0f4864d4f86ffaa8bd74c0471` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.279347 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=175e05a0f4864d4f86ffaa8bd74c0471)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.281370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.339002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175e05a0f4864d4f86ffaa8bd74c0471', 'request_id': 'req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:01.350024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175e05a0f4864d4f86ffaa8bd74c0471', 'request_id': 'req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:01.357473 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175e05a0f4864d4f86ffaa8bd74c0471', 'request_id': 'req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:01.369247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] The token cache is being invalidate because project 175e05a0f4864d4f86ffaa8bd74c0471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:01.369725 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:01.371054 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-fb407e10-6409-4aa6-8e4d-44e5ab2ddcbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:01.377229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1355/2740] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:01 2022] DELETE /identity/v3/projects/175e05a0f4864d4f86ffaa8bd74c0471 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:01.379125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.379670 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.379797 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.404184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:01.405677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.485063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1386/2741] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:01.491130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.491431 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.491721 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.512755 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:01.514003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.521191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.522318 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1356/2742] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:01.522554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.526557 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.536233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62b097-a23e-4fde-9521-9fd407a46b07 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.541229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.541825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.542172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.542449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.543090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:01.543406 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.545002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.556777 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1357/2743] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:59:01.561692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.572116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7f9b-2e5d-4c06-ae5f-a2061731cb76 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.577009 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.577639 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.577924 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.578201 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.578939 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.580888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.584119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6847ab-46b4-4541-8800-ad7d6c152d74 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.587628 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.588231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.588509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.589293 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.591068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.600689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.602106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.608996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1358/2744] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:01 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:01.613116 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.613859 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1387/2745] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:01.680388 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90290ff-693b-40f1-aeca-a0aa27aa9dff admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.685266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.685898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.686210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.686485 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.687113 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.689222 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.714462 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1359/2746] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:01 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:01.715853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.727513 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0301338-b540-4df5-a17b-43d2ca125cd8 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.732640 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.733299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.733684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.733967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.734604 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.736380 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.748253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1388/2747] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:01.752323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.762530 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb98f3e-1ad2-405d-9110-de14a488f239 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.767423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.768024 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.768307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.768585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.769249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.770833 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89896d98-49b3-45a8-a1d7-bb325c70d03e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.782299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1360/2748] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:01.786772 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.797019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b192eb7-4754-473d-b12c-8071ee90bc91 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:01.801634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:01.802290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.802572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.802849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] PATH_INFO: `/v3/projects/8f265c8d9aba45948df6c2bffb3c454d/users/a4e6b5d149304e17a03b0ad1923d08a8/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.819959 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f265c8d9aba45948df6c2bffb3c454d, user_id=a4e6b5d149304e17a03b0ad1923d08a8, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:01.822991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:01.837163 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1389/2749] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:01 2022] PUT /identity/v3/projects/8f265c8d9aba45948df6c2bffb3c454d/users/a4e6b5d149304e17a03b0ad1923d08a8/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:01.848391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:01.848996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:01.849297 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:01.877415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] MFA Rules not processed for user `a4e6b5d149304e17a03b0ad1923d08a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:01.878636 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.890128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:01.955520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.957338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:01.970058 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1361/2750] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:01.980891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:01.983410 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:02.004638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:02.037154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d40f9-cb25-471d-92da-a07dafc15e74 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:02.042344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:02.042979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:02.043261 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:02.043534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:02.045343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:02.049412 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f755ba5-8861-4fd6-b796-ee2aaa42698d None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:02.053308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:02.053915 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:02.054194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:02.054467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:02.057042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:02.058729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a89fc-6d4a-4192-b972-9eec32128c2f service placement] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:02.062863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:02.064268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:02.068996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1390/2751] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:02.074472 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1362/2752] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:03.774344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:03.774865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:03.775093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:03.795523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:03.797109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:03.861270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1391/2753] 10.222.0.25 () {58 vars in 1009 bytes} [Wed Aug 31 09:59:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:03.880575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:03.882699 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:03.961618 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a33b61-9a17-42ba-b866-a64006e1c25d service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:03.969078 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:03.969935 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:03.970376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:03.970798 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:03.982671 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:03.984293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:03.993525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1363/2754] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 09:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:59:04.689428 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:04.759065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918420-880b-4f55-9e16-167b46ce8a84 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:04.764043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:04.764764 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:04.766925 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:04.767278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] PATH_INFO: `/v3/users/a4e6b5d149304e17a03b0ad1923d08a8` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:04.768706 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e6b5d149304e17a03b0ad1923d08a8)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:04.770984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:04.802467 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:04.804380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:04.821627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522817e-da03-4edb-9903-2dbcd2d4ae9f service swift] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:04.831906 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:04.832697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:04.834171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:04.835118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] PATH_INFO: `/v3/users/3e33d89eeef74a0fbcb0225ce146c4d3` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:04.836517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e6b5d149304e17a03b0ad1923d08a8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.837007 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e33d89eeef74a0fbcb0225ce146c4d3)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:04.837584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e6b5d149304e17a03b0ad1923d08a8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.839785 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:04.851889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e6b5d149304e17a03b0ad1923d08a8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.862689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e6b5d149304e17a03b0ad1923d08a8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.864768 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:04.873372 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-6d85c15c-6632-4388-847d-193ae215e31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e6b5d149304e17a03b0ad1923d08a8'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.885055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1392/2755] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:04 2022] DELETE /identity/v3/users/a4e6b5d149304e17a03b0ad1923d08a8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:04.887418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:04.887778 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:04.888127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:04.911738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:04.912300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:04.917598 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:04.925032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e33d89eeef74a0fbcb0225ce146c4d3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.941380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e33d89eeef74a0fbcb0225ce146c4d3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.957681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e33d89eeef74a0fbcb0225ce146c4d3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.966083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e33d89eeef74a0fbcb0225ce146c4d3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.966825 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-0eb63cc4-4456-4834-bd69-d0c370017652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e33d89eeef74a0fbcb0225ce146c4d3'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:04.975862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1364/2756] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:04 2022] DELETE /identity/v3/users/3e33d89eeef74a0fbcb0225ce146c4d3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:59:04.977313 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:04.977600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:04.977880 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:04.998461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:04.999943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.013742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1393/2757] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:05.023462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:05.024885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.078123 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1365/2758] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:05.087859 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:05.090047 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.102225 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ec345-1d28-447b-8641-d85c051ec9fe None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:05.157950 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:05.158586 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.158864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.159135 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.166295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e99ad-f7fc-4054-aeab-15ad7575d4da None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:05.168657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:05.170131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:05.170493 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:05.171099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.171383 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.171656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.180799 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1394/2759] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:05.182971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:05.184779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:05.196288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1366/2760] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:05.653323 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:05.655245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.676471 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e0765-88ac-47f5-a99a-4c8bcac8e835 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:05.690252 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:05.690917 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.691238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.691514 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] PATH_INFO: `/v3/projects/f554d6e8617642eb8a5665973f29c570` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.693499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f554d6e8617642eb8a5665973f29c570)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:05.695427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:05.799988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:05.801541 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.803056 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:05.809646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f554d6e8617642eb8a5665973f29c570', 'request_id': 'req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:05.817174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f554d6e8617642eb8a5665973f29c570', 'request_id': 'req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:05.822251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf78bf-393c-4b85-960b-f03fe53554f4 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:05.825791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f554d6e8617642eb8a5665973f29c570', 'request_id': 'req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:05.846306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] The token cache is being invalidate because project f554d6e8617642eb8a5665973f29c570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:05.847343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:05.848043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-3fcdc9fc-5c07-489a-ab5e-00e33ccc5b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:05.849857 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1395/2761] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:05 2022] DELETE /identity/v3/projects/f554d6e8617642eb8a5665973f29c570 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:05.854970 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed65c3-b5f5-45a0-8a69-d28ad4d0d1a7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.855479 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed65c3-b5f5-45a0-8a69-d28ad4d0d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.857552 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed65c3-b5f5-45a0-8a69-d28ad4d0d1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.889686 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-70ed65c3-b5f5-45a0-8a69-d28ad4d0d1a7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:05.890914 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-70ed65c3-b5f5-45a0-8a69-d28ad4d0d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:05.907783 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:05.910061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.910933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.912016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] PATH_INFO: `/v3/projects/8f265c8d9aba45948df6c2bffb3c454d` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.913815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f265c8d9aba45948df6c2bffb3c454d)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:05.916338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:05.919479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:05.968832 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1396/2762] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:05.976318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:05.976590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:05.976870 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:05.989332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f265c8d9aba45948df6c2bffb3c454d', 'request_id': 'req-67cbc303-5714-4e96-8827-906ffe41c89f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.000123 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.001486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.002953 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f265c8d9aba45948df6c2bffb3c454d', 'request_id': 'req-67cbc303-5714-4e96-8827-906ffe41c89f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.014254 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f265c8d9aba45948df6c2bffb3c454d', 'request_id': 'req-67cbc303-5714-4e96-8827-906ffe41c89f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.031140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] The token cache is being invalidate because project 8f265c8d9aba45948df6c2bffb3c454d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:06.032099 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.033039 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-67cbc303-5714-4e96-8827-906ffe41c89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.038744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1367/2763] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:05 2022] DELETE /identity/v3/projects/8f265c8d9aba45948df6c2bffb3c454d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:06.053352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b8f3-bf5d-4816-9e99-9812adf8bd62 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.053797 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b8f3-bf5d-4816-9e99-9812adf8bd62 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.054184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b8f3-bf5d-4816-9e99-9812adf8bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.079946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1397/2764] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:06.090027 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.090294 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9d62b8f3-bf5d-4816-9e99-9812adf8bd62 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.092105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9d62b8f3-bf5d-4816-9e99-9812adf8bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.100372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1368/2765] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:06.101575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff6a3-1e5a-4bba-84b5-f511686afc4f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.106550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.107133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.107418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.107695 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.108291 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.108421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:06.108578 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.108699 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.108858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.110611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.127416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1398/2766] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:59:06.129641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.137661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.138900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.140651 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72133c8-d8a4-45de-aa3c-0c2b370a5727 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.145498 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.146110 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.146408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.146676 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.147474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.149615 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.151527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1369/2767] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:06.152511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.162535 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb196be-57fc-462c-8ac5-c967037fa049 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.167803 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.168495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.168852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.169239 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.169961 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:06.170359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.172085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.184717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1399/2768] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:06.188013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1370/2769] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:06.193024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.193220 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.216921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5e4b-e840-4af9-a130-f15a87a32d47 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.260416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:06.290400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e6f46-f1da-4183-8109-330345fb66d6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.295283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.296004 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.296284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.296620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.297345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.299200 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.300092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.300748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.301066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.301346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.305303 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.308046 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.323759 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1400/2770] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:06.328100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.339999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3a5c-3b52-4167-bd17-7c99e8a9b66b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.344941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.345576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.345863 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.346167 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.346974 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1371/2771] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:06.349915 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.351215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.352088 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.365582 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1401/2772] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:06.369480 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.394758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375281d4-569e-4406-8f7f-cbfc245694dc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.415211 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:06.435248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf91c2d-c6de-4547-8240-d2edf1eb1d23 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.440075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.440694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.440999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.441276 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.448741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.451494 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.488516 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.489475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.490282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.490282 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.490659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.492402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1372/2773] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:06.492534 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.493465 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.506517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf3b-456e-4405-8049-2c751ccbbad0 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.510094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1402/2774] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:06.512174 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.512323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.513244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.513599 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.513945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.514664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.516497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.532739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1373/2775] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:06.533895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.542280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2647-d669-4d11-84f1-34c364821e74 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.546952 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02ef1f-399f-42e1-9652-f9d8fa961916 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.548409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.549298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.549738 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.550194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] PATH_INFO: `/v3/projects/eb143f5de631430b9b39daa6318c0f09/users/afb8f9e08c944cc5bf7777885ababe32/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.560307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.560984 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.561384 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.561678 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.562300 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.564336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.575172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb143f5de631430b9b39daa6318c0f09, user_id=afb8f9e08c944cc5bf7777885ababe32, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.580083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1374/2776] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:06.580235 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1af51f-6982-4149-83c5-7605d3855453 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.581105 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.593805 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1403/2777] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:06 2022] PUT /identity/v3/projects/eb143f5de631430b9b39daa6318c0f09/users/afb8f9e08c944cc5bf7777885ababe32/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:06.605682 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.605975 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.606253 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.610538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839346-5b5c-479a-aa12-59fc0259e1a9 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.637929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] MFA Rules not processed for user `afb8f9e08c944cc5bf7777885ababe32`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.639171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.687898 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.689204 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.689668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.690402 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] PATH_INFO: `/v3/projects/652969f53c4746f0ac3e06c2bf0783cf/users/dd065f00c6554be59c75671de39bd1b7/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.715942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1404/2778] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:06.720172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.722702 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652969f53c4746f0ac3e06c2bf0783cf, user_id=dd065f00c6554be59c75671de39bd1b7, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.725444 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1751e7-e35b-4b48-a9d4-56a7ac807dc2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.732073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabc4d5-0744-4d49-ac9e-7c09099204a8 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.741984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.741984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.741984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.741984 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.742486 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.742985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.747640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1375/2779] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:06 2022] PUT /identity/v3/projects/652969f53c4746f0ac3e06c2bf0783cf/users/dd065f00c6554be59c75671de39bd1b7/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:06.753902 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.754451 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.754889 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.782619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.783955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.785664 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:06.797621 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1405/2780] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:06.801920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.870600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1376/2781] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:06.886012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.892132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:06.940478 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42242e2-5596-449a-a976-c12b99200fd9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.950055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.951094 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.951576 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.952156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.952897 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.956068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.972724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:07.016775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1406/2782] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/users => generated 300 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:07.018233 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.018364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fc472-a75e-46b6-8f72-fc86fcd87e16 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.029389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16519ef-0697-4a13-b60b-1f0d3b93faea admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.034382 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.034996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.035287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.035559 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.036202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.038092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.053254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1407/2783] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:07.054339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.068254 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48958d3-d195-4876-aa0d-2b9222eeee00 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.074087 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.074907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.075262 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.075613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] PATH_INFO: `/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf/users/5cbceb59f7cb477a890fc80c8d18dc91/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.098741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07dbc0a04d654ad9a4d9c8a26e75fdcf, user_id=5cbceb59f7cb477a890fc80c8d18dc91, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.101509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.122193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.122743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.123027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.123246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1408/2784] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:07 2022] PUT /identity/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf/users/5cbceb59f7cb477a890fc80c8d18dc91/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:07.123430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.127657 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.144138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.146546 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.157672 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1377/2785] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:07.209635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb462f-995f-4de0-80b9-d6fce853d512 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.214505 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.215436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.215436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.215678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.216270 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.218024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5695227-eef4-4123-852e-b612bdd8d4e7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.230033 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1409/2786] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:07.234338 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.244766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf457-b280-483d-9157-e72b1e8254b9 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.250405 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.251133 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.251483 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.251762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] PATH_INFO: `/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf/users/5cbceb59f7cb477a890fc80c8d18dc91/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.254127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07dbc0a04d654ad9a4d9c8a26e75fdcf, user_id=5cbceb59f7cb477a890fc80c8d18dc91, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.256791 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.267596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1378/2787] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:07 2022] PUT /identity/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf/users/5cbceb59f7cb477a890fc80c8d18dc91/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:07.279388 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.279760 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.280104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.319673 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] MFA Rules not processed for user `5cbceb59f7cb477a890fc80c8d18dc91`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:07.321421 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.362942 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:07.402473 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1410/2788] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:07.413634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.415700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.514922 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30b32-464d-4db4-a0bf-31eea2077724 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.518955 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.520153 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.520506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.520856 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.540891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.543285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e74cb-62a8-4481-a13a-ec8be4e8de23 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.560158 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1379/2789] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:07.595408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.687528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d93c-1778-4ebd-a8aa-766682291a09 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.693659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.694343 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.694635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.695092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] PATH_INFO: `/v3/users/afb8f9e08c944cc5bf7777885ababe32` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.696907 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb8f9e08c944cc5bf7777885ababe32)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.699189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.769918 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb8f9e08c944cc5bf7777885ababe32'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.770840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb8f9e08c944cc5bf7777885ababe32'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.794548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb8f9e08c944cc5bf7777885ababe32'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.801932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb8f9e08c944cc5bf7777885ababe32'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.811053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-2e159e40-2c4f-4f00-8d6e-e21857e52c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb8f9e08c944cc5bf7777885ababe32'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.822794 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1411/2790] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:07 2022] DELETE /identity/v3/users/afb8f9e08c944cc5bf7777885ababe32 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:07.825700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.828552 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.830195 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.866969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:07.868664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.981159 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.981713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.981890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.982132 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1380/2791] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:07.994328 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.997128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.040550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] MFA Rules not processed for user `dd065f00c6554be59c75671de39bd1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:08.043692 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.064542 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:08.128616 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:08.164462 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1412/2792] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:08.166266 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124e5d5-ead7-4747-9a29-3bd7f33e97b6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.185181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.189132 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.193364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.194083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.194391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.194677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.221458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.222743 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.261969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1381/2793] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:08.314724 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a966c9-a8cf-4247-b247-5dc748994b4d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.319590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.320770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.321307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.322689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.338335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.341008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.353309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1413/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:08.759087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.766206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.788857 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4fe5-be6e-4564-becf-7a539525f140 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.797224 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.798216 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.798589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.799214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] PATH_INFO: `/v3/projects/eb143f5de631430b9b39daa6318c0f09` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.800990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb143f5de631430b9b39daa6318c0f09)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.803908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.881593 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb143f5de631430b9b39daa6318c0f09', 'request_id': 'req-2338be50-e88f-43d7-af85-84fb51338bd7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.891592 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb143f5de631430b9b39daa6318c0f09', 'request_id': 'req-2338be50-e88f-43d7-af85-84fb51338bd7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.899697 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb143f5de631430b9b39daa6318c0f09', 'request_id': 'req-2338be50-e88f-43d7-af85-84fb51338bd7'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.914586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] The token cache is being invalidate because project eb143f5de631430b9b39daa6318c0f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:08.915040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.915897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.920994 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1382/2795] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:08 2022] DELETE /identity/v3/projects/eb143f5de631430b9b39daa6318c0f09 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:08.922446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.934403 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbffce-eea5-4710-b475-82432d107c64 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.996361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.997059 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.997181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.997453 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] PATH_INFO: `/v3/users/5cbceb59f7cb477a890fc80c8d18dc91` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.998547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbceb59f7cb477a890fc80c8d18dc91)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.000506 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.057039 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbceb59f7cb477a890fc80c8d18dc91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.057819 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbceb59f7cb477a890fc80c8d18dc91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.070161 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbceb59f7cb477a890fc80c8d18dc91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.077212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbceb59f7cb477a890fc80c8d18dc91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.084842 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-706f7764-f431-4182-b26b-18ce9fd97221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbceb59f7cb477a890fc80c8d18dc91'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.091660 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1414/2796] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:08 2022] DELETE /identity/v3/users/5cbceb59f7cb477a890fc80c8d18dc91 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:09.391174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.392776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.406079 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338be50-e88f-43d7-af85-84fb51338bd7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.468490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.469325 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.469673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.470019 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] PATH_INFO: `/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.475128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07dbc0a04d654ad9a4d9c8a26e75fdcf)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.475128 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.539664 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07dbc0a04d654ad9a4d9c8a26e75fdcf', 'request_id': 'req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.548934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07dbc0a04d654ad9a4d9c8a26e75fdcf', 'request_id': 'req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.550212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:09.557863 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07dbc0a04d654ad9a4d9c8a26e75fdcf', 'request_id': 'req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.569622 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] The token cache is being invalidate because project 07dbc0a04d654ad9a4d9c8a26e75fdcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:09.569916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.570634 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-899a3e62-40a5-480c-bc06-0f4e0c6b13b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:09.572126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1383/2797] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:09 2022] DELETE /identity/v3/projects/07dbc0a04d654ad9a4d9c8a26e75fdcf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:09.577566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.577927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.578292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.587038 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:09.602288 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:09.617376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.679507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1415/2798] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:09.687187 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.687582 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.687865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.710006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:09.711247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.724130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1384/2799] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:09.725807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.735550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b7ba6-3933-4f33-ad90-f460bc923af0 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.740158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.740818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.741139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.741412 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.742048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:09.742348 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.743954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.759542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1416/2800] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:59:09.764413 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.773972 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625d5ba-38c8-4ba1-b491-467163770e5c None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.779585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.780207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.780490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.780914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.781680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.783888 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.814701 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1385/2801] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:09.818704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.829220 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b3923-a94d-497d-ae31-24f3aff46988 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.888154 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.888774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.889091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.889370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.890024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.891999 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.918679 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1417/2802] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:09.923124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.935443 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895a18-2668-4adf-aed2-76a01f36dcd6 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.940414 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.941299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.941656 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.941934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.942581 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.944364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.954882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1386/2803] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:09.959104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.969645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd684-f5b6-406c-813f-7fc737fe1ed9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.974354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.974961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.975250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.976354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.976973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.978609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568daf5-5db5-4339-b37b-d40f2dd5c3a7 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.990892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1418/2804] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:09.995616 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:10.007323 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803428-93b4-4722-bca3-6b722f8660ea admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:10.013053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:10.013731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.014064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.014348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] PATH_INFO: `/v3/projects/add84cc59e2a43c6be6e7bc659fc6800/users/f8daea57a2974d2cb30ce721ff1b51fc/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.044124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add84cc59e2a43c6be6e7bc659fc6800, user_id=f8daea57a2974d2cb30ce721ff1b51fc, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:10.046876 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:10.059274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1387/2805] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:09 2022] PUT /identity/v3/projects/add84cc59e2a43c6be6e7bc659fc6800/users/f8daea57a2974d2cb30ce721ff1b51fc/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:10.064489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940f5ce-2064-47ed-b5d0-e763bfa9787e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.065001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940f5ce-2064-47ed-b5d0-e763bfa9787e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.065106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940f5ce-2064-47ed-b5d0-e763bfa9787e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.097643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-8940f5ce-2064-47ed-b5d0-e763bfa9787e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:10.099209 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-8940f5ce-2064-47ed-b5d0-e763bfa9787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:10.192491 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1419/2806] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:10.208262 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:10.210589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:10.284724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999586f6-3ba1-4a2d-954e-129bbe1ef271 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:10.289223 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:10.289942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.290261 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.291435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.307352 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:10.308796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a13c8-ac6c-4f2d-a96b-7efb3d27f53d service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:10.318684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1388/2807] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:10.934345 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.935002 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.935517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.956014 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] MFA Rules not processed for user `dd065f00c6554be59c75671de39bd1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:10.957688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:10.963204 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:11.034954 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1420/2808] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:11.136376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.138404 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.138804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.203218 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] MFA Rules not processed for user `f8daea57a2974d2cb30ce721ff1b51fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:11.204510 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:11.292826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1389/2809] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:11.297257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.314460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922427ec-8c9d-46f1-9325-0f8abf4aae1f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.319295 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.319912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.320195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.320508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.321133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.323617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.366175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.370086 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1421/2810] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:11 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:11.370980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.388327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1825f9f-d32f-421b-93b7-7a97ec3bb2d3 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.417938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:11.457691 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edaaf-0b90-4cbc-a227-d7b0caeced76 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.465203 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.466199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.466626 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.466905 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.467600 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.469613 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.481836 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.482614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.482843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.483160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.485420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:11.493858 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1422/2811] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:11 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:11.496199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.497932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.498045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.508057 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1390/2812] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:11.519855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b440abf-e1b0-4259-ae6c-5e27ab5627aa admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.525966 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.526723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.527078 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.527432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.528205 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.529937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.556181 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1423/2813] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:11.558090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.570229 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e0b5-5dd5-46e6-a3d3-818fa57dd23c service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.576453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.577165 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.577450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.577735 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] PATH_INFO: `/v3/projects/5c5354163a234fc980f569145b2a4cec/users/25d06a2792fc4edb935e3c317849c360/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.598022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5354163a234fc980f569145b2a4cec, user_id=25d06a2792fc4edb935e3c317849c360, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.600714 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.632354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1391/2814] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:11 2022] PUT /identity/v3/projects/5c5354163a234fc980f569145b2a4cec/users/25d06a2792fc4edb935e3c317849c360/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:11.637067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.653798 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb356183-fe6c-4377-8717-0cd231489ddc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.732068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.732688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.733003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.733292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.733932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.735544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e99e0-b94d-45da-8051-20d03ddc5bcc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.749148 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1424/2815] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:11.752269 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.763998 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e518e-dd1d-4fbf-adba-9f4333123ec4 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.785684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.786914 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.787205 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.787525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] PATH_INFO: `/v3/projects/5c5354163a234fc980f569145b2a4cec/users/25d06a2792fc4edb935e3c317849c360/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.789613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5354163a234fc980f569145b2a4cec, user_id=25d06a2792fc4edb935e3c317849c360, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.792860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.804429 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1392/2816] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:11 2022] PUT /identity/v3/projects/5c5354163a234fc980f569145b2a4cec/users/25d06a2792fc4edb935e3c317849c360/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:12.731144 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.731678 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.731895 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.767783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:12.787712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] MFA Rules not processed for user `25d06a2792fc4edb935e3c317849c360`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:12.790158 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.884327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1425/2817] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:12.899138 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.900867 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.975862 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03de28-959f-491c-89c0-cb5595eedf44 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.980013 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.980660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.980944 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.981248 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:13.041252 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:13.042733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:13.065169 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1393/2818] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:15.534929 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:15.545367 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411fff2-6ffb-4ed7-b639-1cd4523b3695 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:15.548854 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:15.549492 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:15.549771 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:15.550980 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:15.559698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:15.561150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:15.570284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1426/2819] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:20.713257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:20.714802 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:20.772768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:20.821403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149072-6cd9-4ccd-9821-3a9cb48ba180 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:20.827709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:20.828337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:20.828621 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:20.828900 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:20.830839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:20.843684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:20.845295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:20.856930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1394/2820] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:25.566141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=36 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:27.089873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:27.105890 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf16df-bb3e-4daa-80cd-323ee463a1ec service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:27.109622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:27.110335 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:27.110622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:27.110902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:27.112739 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:27.126874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:27.128622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:27.141162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1427/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:29.835134 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:29.836632 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:29.923836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:29.941586 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397159-1120-4bfa-b94a-6f74abcdc9b6 service nova] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:29.947684 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:29.948377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:29.948732 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:29.949227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:29.951030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:29.970987 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:29.972584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:29.983208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1395/2822] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:30.001445 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8feea1-7861-4fd0-950d-0b5912234f67 service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.006867 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.007501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.007783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.008058 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] PATH_INFO: `/v3/users/f8daea57a2974d2cb30ce721ff1b51fc` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.009776 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8daea57a2974d2cb30ce721ff1b51fc)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.011701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.058151 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8daea57a2974d2cb30ce721ff1b51fc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.059063 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8daea57a2974d2cb30ce721ff1b51fc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.071392 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8daea57a2974d2cb30ce721ff1b51fc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.079092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8daea57a2974d2cb30ce721ff1b51fc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.086622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8daea57a2974d2cb30ce721ff1b51fc'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.092774 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1428/2823] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:29 2022] DELETE /identity/v3/users/f8daea57a2974d2cb30ce721ff1b51fc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:59:30.266085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.278108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a461a-5baf-438c-aae3-4cacbeb35a04 service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.347600 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.348355 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.348698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.349071 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] PATH_INFO: `/v3/projects/add84cc59e2a43c6be6e7bc659fc6800` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.350487 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=add84cc59e2a43c6be6e7bc659fc6800)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.352442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.411154 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add84cc59e2a43c6be6e7bc659fc6800', 'request_id': 'req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.413050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:30.423689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add84cc59e2a43c6be6e7bc659fc6800', 'request_id': 'req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.432210 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add84cc59e2a43c6be6e7bc659fc6800', 'request_id': 'req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.445547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] The token cache is being invalidate because project add84cc59e2a43c6be6e7bc659fc6800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:30.445547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.446385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.448249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1396/2824] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/projects/add84cc59e2a43c6be6e7bc659fc6800 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:59:30.452427 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.453948 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.467831 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b2a00-22a9-4838-b129-725232a29a51 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.533237 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.533952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.534301 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.534646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] PATH_INFO: `/v3/users/25d06a2792fc4edb935e3c317849c360` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.535952 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d06a2792fc4edb935e3c317849c360)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.538012 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.544229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:30.598338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d06a2792fc4edb935e3c317849c360'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.599006 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d06a2792fc4edb935e3c317849c360'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.611561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d06a2792fc4edb935e3c317849c360'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.619923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d06a2792fc4edb935e3c317849c360'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.629188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e59a2906-3dc5-4438-a7ba-242aaae61ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d06a2792fc4edb935e3c317849c360'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.638818 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1429/2825] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/users/25d06a2792fc4edb935e3c317849c360 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:30.829971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.831596 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.845161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e7ce-e91a-4d09-89f3-2b8b8509f8aa admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.909908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.910784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.911070 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.911348 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] PATH_INFO: `/v3/projects/5c5354163a234fc980f569145b2a4cec` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.912482 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5354163a234fc980f569145b2a4cec)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.914585 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.963468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5354163a234fc980f569145b2a4cec', 'request_id': 'req-8ce59290-a5a4-4142-9721-f0c4156c631f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.971930 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5354163a234fc980f569145b2a4cec', 'request_id': 'req-8ce59290-a5a4-4142-9721-f0c4156c631f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.978971 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5354163a234fc980f569145b2a4cec', 'request_id': 'req-8ce59290-a5a4-4142-9721-f0c4156c631f'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.990630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] The token cache is being invalidate because project 5c5354163a234fc980f569145b2a4cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:30.990968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.991569 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-8ce59290-a5a4-4142-9721-f0c4156c631f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.993090 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1397/2826] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/projects/5c5354163a234fc980f569145b2a4cec => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:30.998093 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.998617 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.998775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.022177 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:31.023460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:31.099812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1430/2827] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:31.101553 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.101897 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.102319 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.124718 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:31.126309 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:31.137004 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1398/2828] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:31.140888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.152726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17a58f-0058-4ac6-8994-3f880a014c6d None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.160732 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.162508 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.162792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.163051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.164062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:31.164430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.166565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.181219 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1431/2829] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:59:31.185577 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.198506 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a4c3-5ff8-427b-b39e-51339dc29954 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.204302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.205027 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.205428 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.206188 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.206631 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.208643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.250511 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1399/2830] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:31.252263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.266140 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759853f2-bf57-47fb-ba5f-9f9d62e7bca4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.382379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.383043 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.383584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.383864 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.385084 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.387022 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.411090 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1432/2831] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:31.415100 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.426008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6666c2c-d9a0-42c9-b67a-48d3be4ab937 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.431103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.431724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.432001 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.432275 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.432945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.434608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.449033 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1400/2832] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:31.450548 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.455807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:31.461580 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9998a44-5e68-4f97-8895-435d25c9d706 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.468927 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.469899 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.470390 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.470777 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.471684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.473313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fe47f-d32a-4e4b-9868-f1b823c25574 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.486054 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1433/2833] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:31.490644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.501110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7336c-9834-486a-804e-96a11e0d2c50 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.505745 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.506393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.506673 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.506947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] PATH_INFO: `/v3/projects/fd2dd1528fbd4cca8db067e6dec98ce5/users/b99f6bf00eb84bd28abd086e2ce45a37/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.523137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2dd1528fbd4cca8db067e6dec98ce5, user_id=b99f6bf00eb84bd28abd086e2ce45a37, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.526457 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.551061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1401/2834] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:31 2022] PUT /identity/v3/projects/fd2dd1528fbd4cca8db067e6dec98ce5/users/b99f6bf00eb84bd28abd086e2ce45a37/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:31.563032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.563317 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.563596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.614065 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] MFA Rules not processed for user `b99f6bf00eb84bd28abd086e2ce45a37`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:31.615364 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:31.704979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1434/2835] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:31.709765 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.716137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:31.780920 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f226c-eeca-4a01-9ed0-c1b0a44dad6e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.785554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.786159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.786438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.786711 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.787342 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.789259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.815575 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1402/2836] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:31.819662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.830744 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672aaeef-16de-4af2-b730-27abcca74d6b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.885741 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.886344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.886618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.886888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.887509 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.889446 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.911333 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1435/2837] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:31 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:31.915995 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.926289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d178a-a1fb-4d55-ad9e-bffd0ecf98c3 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.931722 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.932322 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.932620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.932872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.933608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.935189 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.945614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1403/2838] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:31.949556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.959241 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea352485-2f93-41db-a9e6-3979734918a9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.963795 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.964379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.964658 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.964932 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:31.965584 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:31.967169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b024ac-0edc-4f2e-87bb-4bc27c7a4e08 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:31.976791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1436/2839] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:31.981142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:31.991209 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26712e11-ea0e-46ef-b106-0af6815b6a47 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:31.996372 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:31.997049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:31.997334 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:31.997613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] PATH_INFO: `/v3/projects/846f78b946a0453a894528a973b09c7a/users/18e42950ce3541c4a85a0d1f8e5b9b27/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.011738 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846f78b946a0453a894528a973b09c7a, user_id=18e42950ce3541c4a85a0d1f8e5b9b27, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.014469 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.026609 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1404/2840] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:31 2022] PUT /identity/v3/projects/846f78b946a0453a894528a973b09c7a/users/18e42950ce3541c4a85a0d1f8e5b9b27/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:32.036662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.037187 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.037290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.065546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] MFA Rules not processed for user `18e42950ce3541c4a85a0d1f8e5b9b27`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:32.066769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:32.112517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:32.131684 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1437/2841] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:32.136207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.196975 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee4d5c-334d-4abb-9930-86a2087581d7 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.201594 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.202191 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.202474 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.202748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.203370 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.205285 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.232061 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1405/2842] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:32 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:32.236055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.247246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56b56-cc1c-49fb-ad56-a41a577ad1dc None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.303232 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.303761 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.304040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.304314 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.304937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.306944 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.327296 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1438/2843] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:32 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:32.331376 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.341835 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab586a1-b68c-4ae8-ba26-d38b910eaeef admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.346337 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.346923 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.347199 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.347468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.348093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.349761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.360137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1406/2844] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:32.364162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.374720 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7ec1f-4d94-4f07-be19-baba0bb2094b admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.380355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.381141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.381420 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.381693 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] PATH_INFO: `/v3/projects/6f41ffc9204a44159394785ab3552b75/users/186255531f86446b95c18a7f6db39597/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.397507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f41ffc9204a44159394785ab3552b75, user_id=186255531f86446b95c18a7f6db39597, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.400195 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.415005 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1439/2845] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:32 2022] PUT /identity/v3/projects/6f41ffc9204a44159394785ab3552b75/users/186255531f86446b95c18a7f6db39597/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:32.418988 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.429301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea558-cd6e-4210-8e6b-a0536fda2064 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.455824 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:32.487820 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.488498 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.488748 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.489050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.489686 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.491255 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f313c7f-268f-480a-bcff-180bfa95452e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.501796 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1407/2846] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:32.505957 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.517436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4995a0-6362-4217-8219-40ecca761043 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.522003 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.522641 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.522916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.523190 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] PATH_INFO: `/v3/projects/6f41ffc9204a44159394785ab3552b75/users/186255531f86446b95c18a7f6db39597/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.525844 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f41ffc9204a44159394785ab3552b75, user_id=186255531f86446b95c18a7f6db39597, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.528457 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.538418 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1440/2847] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:32 2022] PUT /identity/v3/projects/6f41ffc9204a44159394785ab3552b75/users/186255531f86446b95c18a7f6db39597/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:32.548619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.548911 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.549226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.577700 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] MFA Rules not processed for user `186255531f86446b95c18a7f6db39597`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:32.578846 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:32.645515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1408/2848] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:32.656530 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:32.658247 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:32.716162 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf3c5-b644-4eae-9c41-9996012fa152 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:32.720032 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:32.720642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:32.720976 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:32.721258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:32.776408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:32.778891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:32.780402 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:32.791312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1441/2849] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:34.297842 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:34.309104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5961-bcd9-4dd8-92ef-521d0b5575d8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:34.313437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:34.314135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:34.314479 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:34.314815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:34.387385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:34.388377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:34.398625 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1409/2850] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:34.958263 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:34.966775 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03922-19bc-4147-aeeb-cad952b0723d service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:34.970292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:34.970891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:34.971165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:34.971432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:34.980520 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:34.981973 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961421d2-1b74-4249-a185-498f70cb31eb service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:34.992085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1442/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:38.872076 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:38.922207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:38.936521 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cff23-dcd1-4214-85c1-b5140ca3ea53 service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:38.941448 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:38.941693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:38.941977 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:38.942316 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] PATH_INFO: `/v3/users/b99f6bf00eb84bd28abd086e2ce45a37` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:38.943368 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99f6bf00eb84bd28abd086e2ce45a37)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:38.945310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:38.999481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99f6bf00eb84bd28abd086e2ce45a37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.010014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99f6bf00eb84bd28abd086e2ce45a37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.022940 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99f6bf00eb84bd28abd086e2ce45a37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.030614 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99f6bf00eb84bd28abd086e2ce45a37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.031012 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99f6bf00eb84bd28abd086e2ce45a37'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.036847 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1410/2852] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:38 2022] DELETE /identity/v3/users/b99f6bf00eb84bd28abd086e2ce45a37 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:39.042823 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:39.043441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:39.043441 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:39.063422 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:39.064677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:39.129299 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1443/2853] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:39.139075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:39.140527 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:39.198341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d058052-cc03-495e-94a5-3aa2d736d06d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:39.201964 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:39.202566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:39.202852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:39.203170 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:39.213117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:39.214651 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:39.225075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1411/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:39.790728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:39.792354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:39.806246 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fde3d-ebbb-4259-855c-8ae2bcc9ca1f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:39.811010 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:39.811788 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:39.812141 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:39.812512 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] PATH_INFO: `/v3/projects/fd2dd1528fbd4cca8db067e6dec98ce5` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:39.813735 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2dd1528fbd4cca8db067e6dec98ce5)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:39.815688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:39.868882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2dd1528fbd4cca8db067e6dec98ce5', 'request_id': 'req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.875481 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2dd1528fbd4cca8db067e6dec98ce5', 'request_id': 'req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.882313 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2dd1528fbd4cca8db067e6dec98ce5', 'request_id': 'req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.895344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] The token cache is being invalidate because project fd2dd1528fbd4cca8db067e6dec98ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:39.895571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.896180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:39.897668 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1444/2855] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:39 2022] DELETE /identity/v3/projects/fd2dd1528fbd4cca8db067e6dec98ce5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:39.901646 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:39.911098 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db2b37-78d6-4231-bf70-7b7b3a1d3e04 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:39.970002 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:39.970647 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:39.970937 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:39.971212 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] PATH_INFO: `/v3/users/18e42950ce3541c4a85a0d1f8e5b9b27` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:39.972397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e42950ce3541c4a85a0d1f8e5b9b27)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:39.974360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:39.988107 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:40.023233 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e42950ce3541c4a85a0d1f8e5b9b27'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.030317 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e42950ce3541c4a85a0d1f8e5b9b27'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.044874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e42950ce3541c4a85a0d1f8e5b9b27'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.051566 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e42950ce3541c4a85a0d1f8e5b9b27'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.052548 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e42950ce3541c4a85a0d1f8e5b9b27'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.058877 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1412/2856] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:39 2022] DELETE /identity/v3/users/18e42950ce3541c4a85a0d1f8e5b9b27 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:40.334701 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:40.336327 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:40.350414 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14dd17-dd4e-4a44-9aab-1b9ee66cb90f admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:40.409351 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:40.409913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:40.410196 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:40.410472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] PATH_INFO: `/v3/projects/846f78b946a0453a894528a973b09c7a` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:40.411591 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=846f78b946a0453a894528a973b09c7a)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:40.413554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:40.421594 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:40.461571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846f78b946a0453a894528a973b09c7a', 'request_id': 'req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.467810 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846f78b946a0453a894528a973b09c7a', 'request_id': 'req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.474098 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846f78b946a0453a894528a973b09c7a', 'request_id': 'req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.487611 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] The token cache is being invalidate because project 846f78b946a0453a894528a973b09c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:40.487902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.488535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.490060 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1445/2857] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:40 2022] DELETE /identity/v3/projects/846f78b946a0453a894528a973b09c7a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:40.494626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:40.496062 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:40.509164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815ad9-f9fa-4b94-a036-1d46749abb63 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:40.566117 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:40.566766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:40.567048 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:40.567326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] PATH_INFO: `/v3/users/186255531f86446b95c18a7f6db39597` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:40.568458 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=186255531f86446b95c18a7f6db39597)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:40.570398 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:40.618982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186255531f86446b95c18a7f6db39597'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.626292 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186255531f86446b95c18a7f6db39597'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.638567 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186255531f86446b95c18a7f6db39597'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.645400 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186255531f86446b95c18a7f6db39597'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.646056 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6bcc662d-8cdc-48f1-9d5f-0c58eefdec1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186255531f86446b95c18a7f6db39597'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:40.651852 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1413/2858] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:40 2022] DELETE /identity/v3/users/186255531f86446b95c18a7f6db39597 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:40.938413 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:40.939838 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:40.952871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b287e7-39ed-41b5-90c9-7e681ef86d7a admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.017055 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.017700 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.017985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.018264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] PATH_INFO: `/v3/projects/6f41ffc9204a44159394785ab3552b75` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.019377 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f41ffc9204a44159394785ab3552b75)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.021341 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.068643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f41ffc9204a44159394785ab3552b75', 'request_id': 'req-edae4038-80e2-41f0-b496-9f209d61f4a5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.074719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f41ffc9204a44159394785ab3552b75', 'request_id': 'req-edae4038-80e2-41f0-b496-9f209d61f4a5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.080865 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f41ffc9204a44159394785ab3552b75', 'request_id': 'req-edae4038-80e2-41f0-b496-9f209d61f4a5'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.095257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] The token cache is being invalidate because project 6f41ffc9204a44159394785ab3552b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:41.095549 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.096165 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-edae4038-80e2-41f0-b496-9f209d61f4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.097717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1446/2859] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:40 2022] DELETE /identity/v3/projects/6f41ffc9204a44159394785ab3552b75 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:41.103887 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.104397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.104397 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.124766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:41.126017 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.130393 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:41.185737 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1414/2860] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:41.249723 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.250007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.250283 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.270307 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:41.271507 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.279391 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1447/2861] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:41.283459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.293250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d471d5f-1d58-445b-83af-93720d28d981 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.298551 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.299180 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.299481 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.299778 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.300438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:41.300740 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.302661 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.315332 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1415/2862] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:41.319642 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.329432 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a7be2-5b4c-4ded-a04a-144c428fc196 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.334066 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.334659 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.334939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.335472 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.335813 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.337786 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.358107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:41.366671 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1448/2863] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:41 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:41.371087 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.381026 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69f330-3d6e-4170-b337-cfedd873de5e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.442767 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.443302 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.443589 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.443891 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.444538 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.447311 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.468490 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1416/2864] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:41 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:41.472598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.487672 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea14a5-6779-4f64-8ffd-bc36761b28b4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.493376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.495728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.496018 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.496740 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.497565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.500164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.510920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1449/2865] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:41.515104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.526268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e58e1-08f6-46c5-b8ac-54292a6f520f admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.531124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.531761 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.532072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.532346 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.533014 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.534626 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43a97e-7ba7-4d83-8e97-1f1afd01a064 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.546245 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1417/2866] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:41.550089 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.561275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fce8f0-fad1-4ef9-a057-7efaf8593abd admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.565997 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.566652 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.566936 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.567212 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] PATH_INFO: `/v3/projects/63b016dc478a4e99babf042ca32cff71/users/7c5168c5f7ae4f4381e70c7b04d385b5/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.582622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b016dc478a4e99babf042ca32cff71, user_id=7c5168c5f7ae4f4381e70c7b04d385b5, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.585292 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.597845 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1450/2867] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:41 2022] PUT /identity/v3/projects/63b016dc478a4e99babf042ca32cff71/users/7c5168c5f7ae4f4381e70c7b04d385b5/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:41.608450 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.608746 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.609065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.636817 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] MFA Rules not processed for user `7c5168c5f7ae4f4381e70c7b04d385b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:41.638308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.705184 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1418/2868] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:41.717008 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.719298 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.780625 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300064fe-e0af-4cee-aaed-030d06c16545 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.784243 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.784852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.785164 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.785438 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.794108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.795532 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d30a03-070e-423a-ac07-6e4b32d7eb1f service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.804431 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1451/2869] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:41.831620 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.836985 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:41.913565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe833a-e23d-46fc-bf07-6dd08d9bdc77 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.913565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.913565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.913565 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.913965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] PATH_INFO: `/v3/users/7c5168c5f7ae4f4381e70c7b04d385b5` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.913965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5168c5f7ae4f4381e70c7b04d385b5)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.913965 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.966641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5168c5f7ae4f4381e70c7b04d385b5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.974354 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5168c5f7ae4f4381e70c7b04d385b5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:41.997929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5168c5f7ae4f4381e70c7b04d385b5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.005174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5168c5f7ae4f4381e70c7b04d385b5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.005997 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5168c5f7ae4f4381e70c7b04d385b5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.012172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1419/2870] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:41 2022] DELETE /identity/v3/users/7c5168c5f7ae4f4381e70c7b04d385b5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:42.016546 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.016928 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.017474 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.165489 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:42.166807 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:42.226439 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1452/2871] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:42.236260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:42.237694 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:42.298008 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb55fd-1ce0-4ac6-a725-e6e8ed4e694b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:42.301668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:42.302547 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.302826 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.303092 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.312874 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:42.314306 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cb4f9-cc2b-4fd0-b6f8-f975be24f7a7 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:42.324034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1420/2872] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:42.600587 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:42.602572 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:42.614972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d346eda-6645-47b0-aee1-105f2443d3d7 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:42.620371 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:42.621092 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.621375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.621646 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] PATH_INFO: `/v3/projects/63b016dc478a4e99babf042ca32cff71` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.622758 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b016dc478a4e99babf042ca32cff71)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:42.624656 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:42.670689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b016dc478a4e99babf042ca32cff71', 'request_id': 'req-e756b740-c12c-446a-a46a-d319de08b56a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.676111 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b016dc478a4e99babf042ca32cff71', 'request_id': 'req-e756b740-c12c-446a-a46a-d319de08b56a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.678332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:42.683869 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b016dc478a4e99babf042ca32cff71', 'request_id': 'req-e756b740-c12c-446a-a46a-d319de08b56a'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.696588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] The token cache is being invalidate because project 63b016dc478a4e99babf042ca32cff71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:42.696876 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.697545 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e756b740-c12c-446a-a46a-d319de08b56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:42.699019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1453/2873] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:42 2022] DELETE /identity/v3/projects/63b016dc478a4e99babf042ca32cff71 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:42.703768 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.704075 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.704326 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.725804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:42.726932 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:42.747894 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:42.789452 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1421/2874] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:42.797588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.797879 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.798156 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.817538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:42.818734 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:42.826496 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1454/2875] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:42.830742 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:42.841251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320e258-148a-4a5a-bd0b-1fc655089d88 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:42.846010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:42.846615 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.846899 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.847174 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] PATH_INFO: `/v3/domains` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.847843 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88241) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:42.848148 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:42.849776 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:42.861739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1422/2876] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:42.867475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:42.876575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92749-06bc-4603-928c-0f0d23ef622e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:42.881239 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:42.881840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.882125 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.882396 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.883026 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:42.884979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:42.911960 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1455/2877] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:42 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:42.916809 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:42.927305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a7f07-fb8c-41b9-be0f-2657e64ed383 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:42.987669 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:42.988288 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:42.988568 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:42.988850 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] PATH_INFO: `/v3/users` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:42.989517 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:42.991423 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.016604 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1423/2878] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:42 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:43.020809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.044273 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078726-8531-4477-8468-a223b493f0e4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.049704 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.050416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.050934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.051304 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.052306 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.054051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.065245 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1456/2879] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:43.069795 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.080484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beb9db-4c2c-47ee-9981-2fd546e9ca3d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.085244 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.085895 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.086208 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.086590 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.087341 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.088919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e0661-a266-4fa9-ac05-877bf207a792 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.108066 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1424/2880] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:43.112201 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.122361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67f73-2e60-4b97-bcc4-0ad8b8dd02d9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.126937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.127593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.127875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.128145 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] PATH_INFO: `/v3/projects/7d453cee087a4825a372608f2059d08f/users/71af7cfbb80149c498103aea7239e654/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.144836 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d453cee087a4825a372608f2059d08f, user_id=71af7cfbb80149c498103aea7239e654, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.147521 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.162287 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1457/2881] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:43 2022] PUT /identity/v3/projects/7d453cee087a4825a372608f2059d08f/users/71af7cfbb80149c498103aea7239e654/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:43.173183 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.173561 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.173905 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.204146 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] MFA Rules not processed for user `71af7cfbb80149c498103aea7239e654`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:43.205396 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.276667 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1425/2882] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:43.284381 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.286208 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.288217 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.289815 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.316855 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:43.348168 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae520a-0390-4023-9c25-ab2ce4a529e9 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.349345 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d240bb0-3fff-4ab1-9ffc-322e5c463481 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.351835 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.352425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.352702 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.352985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.353106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.353706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.353992 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.354268 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.354877 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.362641 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.364041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.374698 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1426/2883] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:43.422476 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.423921 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bf65e-929e-4392-b582-17bc339dff10 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.434426 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1458/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:53.370377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=16 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:53.429318 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:57.593273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:57.655741 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4b21-b138-4da1-9bf5-42c46092e65f service cinder] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:57.660108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:57.660733 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:57.661110 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:57.661389 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] PATH_INFO: `/v3/users/71af7cfbb80149c498103aea7239e654` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:57.662504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=71af7cfbb80149c498103aea7239e654)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:57.664919 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:57.715161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71af7cfbb80149c498103aea7239e654'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:57.723415 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71af7cfbb80149c498103aea7239e654'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:57.736257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71af7cfbb80149c498103aea7239e654'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:57.743065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71af7cfbb80149c498103aea7239e654'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:57.743713 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71af7cfbb80149c498103aea7239e654'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:57.749640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1427/2885] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:57 2022] DELETE /identity/v3/users/71af7cfbb80149c498103aea7239e654 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:57.754104 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:57.754609 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:57.754729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:57.773955 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:57.775180 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:57.847416 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1459/2886] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 09:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:57.857226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:57.858793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:57.917617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293deec-dc71-4926-8535-6688fe4fa85d admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:57.921198 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:57.921792 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:57.922068 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:57.922340 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:57.932130 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:57.933595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:57.943029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1428/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:58.209052 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:58.210517 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:58.222934 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaede41-0615-4e00-949f-3a67dfd5da4f None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:58.227544 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:58.228354 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:58.228638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:58.228912 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] PATH_INFO: `/v3/projects/7d453cee087a4825a372608f2059d08f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:58.230067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d453cee087a4825a372608f2059d08f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:58.231985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:58.280169 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d453cee087a4825a372608f2059d08f', 'request_id': 'req-4e1cff41-22be-4a57-b459-2f89ad323516'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:58.285790 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d453cee087a4825a372608f2059d08f', 'request_id': 'req-4e1cff41-22be-4a57-b459-2f89ad323516'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:58.292477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d453cee087a4825a372608f2059d08f', 'request_id': 'req-4e1cff41-22be-4a57-b459-2f89ad323516'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:58.305425 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] The token cache is being invalidate because project 7d453cee087a4825a372608f2059d08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:58.305843 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:58.306523 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:58.308076 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1460/2888] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:58 2022] DELETE /identity/v3/projects/7d453cee087a4825a372608f2059d08f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:07.938754 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:08.301742 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:11.564709 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.566172 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.578391 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ace5e-ee7e-4b4a-95d0-3eeca3b24129 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.627437 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.628074 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.628361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.628638 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.630421 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.694251 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.695681 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.705312 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1429/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:17.937719 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.939188 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:17.953223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cff41-22be-4a57-b459-2f89ad323516 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.957961 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.958698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.959040 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.959436 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] PATH_INFO: `/v3/users/5669f5e5dcd74e01b4fb7c1bd512bd82` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.975789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5669f5e5dcd74e01b4fb7c1bd512bd82)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.977749 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.022171 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5669f5e5dcd74e01b4fb7c1bd512bd82'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.022814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5669f5e5dcd74e01b4fb7c1bd512bd82'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.034568 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5669f5e5dcd74e01b4fb7c1bd512bd82'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.041370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5669f5e5dcd74e01b4fb7c1bd512bd82'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.050638 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5669f5e5dcd74e01b4fb7c1bd512bd82'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.057465 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1461/2890] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:17 2022] DELETE /identity/v3/users/5669f5e5dcd74e01b4fb7c1bd512bd82 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:18.233085 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.243736 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ba3e-cad4-4e2f-ad5e-bbc15830d19d service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.306660 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.307293 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.307573 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.307848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] PATH_INFO: `/v3/projects/9df6e3fe84eb42b4a1a70e76b5de9fa2` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.318644 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df6e3fe84eb42b4a1a70e76b5de9fa2)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.321272 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.365869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:18.370619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df6e3fe84eb42b4a1a70e76b5de9fa2', 'request_id': 'req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.379173 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df6e3fe84eb42b4a1a70e76b5de9fa2', 'request_id': 'req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.386643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df6e3fe84eb42b4a1a70e76b5de9fa2', 'request_id': 'req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.399143 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] The token cache is being invalidate because project 9df6e3fe84eb42b4a1a70e76b5de9fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:18.399436 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.400137 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.401790 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1430/2891] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:18 2022] DELETE /identity/v3/projects/9df6e3fe84eb42b4a1a70e76b5de9fa2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:18.405855 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.407193 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:18.420149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6edb68-2956-447d-9046-d25273025b80 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.475986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.476607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.476892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.477198 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] PATH_INFO: `/v3/users/fd3a2c62a61a4652ad3f70e70788470f` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.491192 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3a2c62a61a4652ad3f70e70788470f)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.493139 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.537001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3a2c62a61a4652ad3f70e70788470f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.537690 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3a2c62a61a4652ad3f70e70788470f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.542053 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:18.551937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3a2c62a61a4652ad3f70e70788470f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.558824 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3a2c62a61a4652ad3f70e70788470f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.565561 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3a2c62a61a4652ad3f70e70788470f'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.571814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1462/2892] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:18 2022] DELETE /identity/v3/users/fd3a2c62a61a4652ad3f70e70788470f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:18.746127 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.747564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:18.762082 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10114a7b-f5de-4a43-8ebc-6b2d68f8cbd2 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.819093 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.819727 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.820018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.820289 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] PATH_INFO: `/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.830558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb198f9758f4ad5af15ced1455ec6e0)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.832692 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.880819 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb198f9758f4ad5af15ced1455ec6e0', 'request_id': 'req-b653b173-38d6-4f32-b517-11064ed8b1de'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.889613 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb198f9758f4ad5af15ced1455ec6e0', 'request_id': 'req-b653b173-38d6-4f32-b517-11064ed8b1de'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.896307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb198f9758f4ad5af15ced1455ec6e0', 'request_id': 'req-b653b173-38d6-4f32-b517-11064ed8b1de'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.906938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] The token cache is being invalidate because project ebb198f9758f4ad5af15ced1455ec6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:18.907227 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.907845 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b653b173-38d6-4f32-b517-11064ed8b1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:18.909339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1431/2893] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:18 2022] DELETE /identity/v3/projects/ebb198f9758f4ad5af15ced1455ec6e0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:28.567986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:DELETE=3,keystone:SELECT=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:28.903908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:32.284698 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:32.286131 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:32.360408 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457c7f-35c0-4c63-9566-230b1dd727e4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:32.417762 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:32.418368 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:32.418648 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:32.418920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:32.420622 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:32.431766 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:32.433378 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:32.443186 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1463/2894] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:42.438817 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=58 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:54.860823 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:54.861380 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:54.861495 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:54.880675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] MFA Rules not processed for user `dd065f00c6554be59c75671de39bd1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:54.882006 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:54.936120 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1432/2895] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:01:04.929350 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=25 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:29.906883 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:29.908258 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:29.963849 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf582-b590-4da6-aa29-497b761a7cc3 service placement] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:29.967386 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1085dcce677340bca92684d362e2fe7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:29.967979 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:29.968257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:29.968535 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:29.970231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:30.038752 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:30.040202 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:30.051375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1464/2896] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:33.688427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:33.689873 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:33.745558 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc1f4d-9a89-4f7e-837e-6bb86a2e6a80 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:33.749081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:33.749689 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:33.749973 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:33.750249 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:33.751945 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:33.765025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:33.766442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:33.777708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1433/2897] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:40.046338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=54 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:43.772861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=26 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:02.994785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:02.996728 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.063888 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f35a-1ff2-470e-bfeb-29bdb2822fb4 service cinder] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.068375 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.069048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.069320 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.069590 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] PATH_INFO: `/v3/users/dd065f00c6554be59c75671de39bd1b7` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.070703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd065f00c6554be59c75671de39bd1b7)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.072612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.124680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd065f00c6554be59c75671de39bd1b7'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.125361 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd065f00c6554be59c75671de39bd1b7'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.137571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd065f00c6554be59c75671de39bd1b7'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.144309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd065f00c6554be59c75671de39bd1b7'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.151571 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-8bed6e1c-c82d-4f05-a738-e581156d529c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd065f00c6554be59c75671de39bd1b7'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.157687 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1465/2898] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:02:02 2022] DELETE /identity/v3/users/dd065f00c6554be59c75671de39bd1b7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.337643 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.350040 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0de50-f51a-4702-9b35-91d00b0f041b service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.413049 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.413683 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.413969 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.414247 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] PATH_INFO: `/v3/projects/652969f53c4746f0ac3e06c2bf0783cf` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.415367 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] RBAC: Authorizing `identity:delete_project(project_id=652969f53c4746f0ac3e06c2bf0783cf)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.417301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.463921 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652969f53c4746f0ac3e06c2bf0783cf', 'request_id': 'req-b4ce4710-adab-44f7-bf48-5b4fd75e217c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.471193 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652969f53c4746f0ac3e06c2bf0783cf', 'request_id': 'req-b4ce4710-adab-44f7-bf48-5b4fd75e217c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.477461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652969f53c4746f0ac3e06c2bf0783cf', 'request_id': 'req-b4ce4710-adab-44f7-bf48-5b4fd75e217c'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.488065 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] The token cache is being invalidate because project 652969f53c4746f0ac3e06c2bf0783cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88241) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:02:03.488403 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.489161 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-b4ce4710-adab-44f7-bf48-5b4fd75e217c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.490680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1434/2899] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:02:03 2022] DELETE /identity/v3/projects/652969f53c4746f0ac3e06c2bf0783cf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.496575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.496874 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.497173 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.516519 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:03.517689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.576309 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1466/2900] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:02:03.584064 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.584359 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.584640 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.606274 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:03.607478 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.615407 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1435/2901] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:03.619588 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.629073 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c2319-45a4-42e7-a8cd-fb1b46e30a0b None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.633477 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.634061 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.634338 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.634606 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.635238 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:02:03.635565 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.637172 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.648257 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1467/2902] 10.222.0.25 () {58 vars in 1153 bytes} [Wed Aug 31 10:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:02:03.652860 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.662032 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998513f-f658-4d5b-8780-7df308225fc7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.666619 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.667207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.667484 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.668364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.669003 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.670882 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.696675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1436/2903] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 10:02:03 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:02:03.700643 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.710242 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960707-0435-4ad6-811f-ca5a4e9c8aa8 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.713722 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:03.768891 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.769452 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.769747 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.770001 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.770620 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.772563 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.794234 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1468/2904] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 10:02:03 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:02:03.798762 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.809011 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d1d8-3f37-438e-88aa-68ccd4ae3a9e admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.813677 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.814271 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.814554 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.814829 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.815461 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.817119 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.827491 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1437/2905] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 10:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:02:03.831336 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.842045 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51305da1-a050-4181-98a9-a87a5468d8fc admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.846866 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.847460 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.847745 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.848020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] PATH_INFO: `/v3/roles` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.848688 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.850353 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215293b-edbc-4a03-9cc7-5088a68dde18 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.861389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1469/2906] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 10:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:02:03.865459 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.875277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47346cb9-8c19-4190-b96a-00ac979b289b admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.879861 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.880499 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.880779 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.881106 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] PATH_INFO: `/v3/projects/52d82faa2ab445e18d6e3d668821c1e5/users/e170e5057c6f4467b247b2175d2b3ba5/roles/5aa5cafea64a411885edceea9373b3df` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.896308 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d82faa2ab445e18d6e3d668821c1e5, user_id=e170e5057c6f4467b247b2175d2b3ba5, role_id=5aa5cafea64a411885edceea9373b3df)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.898962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.912016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1438/2907] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 10:02:03 2022] PUT /identity/v3/projects/52d82faa2ab445e18d6e3d668821c1e5/users/e170e5057c6f4467b247b2175d2b3ba5/roles/5aa5cafea64a411885edceea9373b3df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.917042 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d60d8-f52d-4037-a2ec-a6cb5e3fb11b None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.917332 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d60d8-f52d-4037-a2ec-a6cb5e3fb11b None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.917618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d60d8-f52d-4037-a2ec-a6cb5e3fb11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.936937 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-ad9d60d8-f52d-4037-a2ec-a6cb5e3fb11b None None] MFA Rules not processed for user `3c3232bfea3b429697a9f4a4bfb3705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:03.938160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ad9d60d8-f52d-4037-a2ec-a6cb5e3fb11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.995499 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1470/2908] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:04.005160 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:04.006913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:04.038449 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:04.067067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e7911-ace7-470c-9f96-ae66152343b8 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:04.071430 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:04.072041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:04.072324 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:04.072602 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:04.082420 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:04.083844 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:04.093962 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1439/2909] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:08.080769 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.081278 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.081393 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.110873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] MFA Rules not processed for user `e170e5057c6f4467b247b2175d2b3ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:08.112105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:08.179525 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1471/2910] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:08.183934 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.193571 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef55c0f-5d30-43eb-a385-5cd56b1748ab service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.198202 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.198821 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.199108 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.199386 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] PATH_INFO: `/v3/projects` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.200034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.201996 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.232260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1440/2911] 10.222.0.25 () {60 vars in 1153 bytes} [Wed Aug 31 10:02:08 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:02:08.236376 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.247308 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd839d0-94fb-4f7b-ab8e-1632feeaca68 None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.258612 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:08.304812 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.305447 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.305727 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.305998 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] PATH_INFO: `/v3/users` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.306618 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.308558 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.329985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1472/2912] 10.222.0.25 () {60 vars in 1144 bytes} [Wed Aug 31 10:02:08 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:02:08.334417 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.346020 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73160a6-c566-40ae-bf96-f54c5e2b327c admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.351982 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.352706 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.353030 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.353908 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.354579 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.356913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.367958 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1441/2913] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:02:08.372099 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.382312 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10320-4496-4af0-b316-89a3c5100b07 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.387430 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.388311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.388598 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.388875 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] PATH_INFO: `/v3/projects/71e85f6e576745cfa30b47b11a9b2f01/users/47d11add4eae4d4986fc8288c22af614/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.406803 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e85f6e576745cfa30b47b11a9b2f01, user_id=47d11add4eae4d4986fc8288c22af614, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.409705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.426100 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1473/2914] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 10:02:08 2022] PUT /identity/v3/projects/71e85f6e576745cfa30b47b11a9b2f01/users/47d11add4eae4d4986fc8288c22af614/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:02:08.432731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.444164 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498025-68be-4082-8ac7-35de41a393f1 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.509307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.509916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.510215 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.510468 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.511605 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.513206 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d12032-32ba-4e04-abf3-1e11421f3f4e admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.523135 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1442/2915] 10.222.0.25 () {58 vars in 1122 bytes} [Wed Aug 31 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:02:08.527930 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:08.538379 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63161-205f-48f5-9db9-2c2c605dc836 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:08.543901 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:08.544573 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:08.544853 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:08.545223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] PATH_INFO: `/v3/projects/71e85f6e576745cfa30b47b11a9b2f01/users/47d11add4eae4d4986fc8288c22af614/roles/3058b83cad264d36bc1ba356d7561bef` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:08.547265 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e85f6e576745cfa30b47b11a9b2f01, user_id=47d11add4eae4d4986fc8288c22af614, role_id=3058b83cad264d36bc1ba356d7561bef)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:08.550300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:08.561916 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1474/2916] 10.222.0.25 () {60 vars in 1483 bytes} [Wed Aug 31 10:02:08 2022] PUT /identity/v3/projects/71e85f6e576745cfa30b47b11a9b2f01/users/47d11add4eae4d4986fc8288c22af614/roles/3058b83cad264d36bc1ba356d7561bef => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:12.827072 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f59-2358-4858-8d09-83c4209520b7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:12.828504 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f59-2358-4858-8d09-83c4209520b7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:12.829166 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f59-2358-4858-8d09-83c4209520b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:12.890929 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-f3ae1f59-2358-4858-8d09-83c4209520b7 None None] MFA Rules not processed for user `47d11add4eae4d4986fc8288c22af614`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:12.892283 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f3ae1f59-2358-4858-8d09-83c4209520b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:12.970693 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1443/2917] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:12.986792 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:12.988991 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:13.082908 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2f4a-95f4-406a-b3ff-f994b1ab85d6 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:13.096072 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:13.096713 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:13.097071 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:13.097350 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:13.158903 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:13.160554 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:13.170160 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1475/2918] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:14.124756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:14.125299 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:14.125408 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:14.146966 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] MFA Rules not processed for user `e170e5057c6f4467b247b2175d2b3ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:14.149041 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:14.158063 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1444/2919] 10.222.0.25 () {58 vars in 958 bytes} [Wed Aug 31 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:23.165816 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:24.139520 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:40.708191 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:40.766020 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae277196-2316-44c6-a78b-cccc6745a83f service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:40.769607 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:40.770218 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:40.770501 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:40.770826 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:40.773635 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:40.785593 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:40.787044 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:40.796157 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1476/2920] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:50.792791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=26 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:57.515442 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:57.516872 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:57.529534 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb2ecb-358a-47e0-853a-a523daaac7e6 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:57.533140 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:57.533739 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:57.534023 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:57.534301 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:57.543868 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:57.545331 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86189e-9ea5-40e1-90ef-c10b73f7fc21 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:57.554948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1445/2921] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:59.334068 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:59.347993 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a33f5-f530-4e06-9ab1-4d7682e8cd86 service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:59.354415 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:59.355199 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:59.355556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:59.355902 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:59.357785 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:59.370240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:59.371851 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:59.382827 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1477/2922] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:03:07.550771 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=6 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:09.378184 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=6 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:21.169022 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:21.169512 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:21.169617 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:21.188916 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] MFA Rules not processed for user `4128d9697d5e4f4b81eb3b46ec7b95b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:03:21.190142 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:21.244564 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1446/2923] 10.222.0.25 () {58 vars in 967 bytes} [Wed Aug 31 10:03:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:03:21.284631 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:21.286108 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:21.354079 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0d18-9543-413e-9976-aa3e6db5a1d2 service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:21.358779 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24659ce97c204823a8968c757ea40e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:21.359370 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:21.359645 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:21.359913 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:21.367683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:21.369300 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:21.377194 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1478/2924] 10.222.0.25 () {60 vars in 1384 bytes} [Wed Aug 31 10:03:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:03:27.913766 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:27.915933 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:27.983680 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd68463-c53b-4213-aedc-3ed9dbf8c1d8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:27.988668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:27.989377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:27.989756 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:27.990053 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] PATH_INFO: `/v3/users/633f1828cf87496fbc0dad682bf9986b` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:27.991505 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633f1828cf87496fbc0dad682bf9986b)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:27.993608 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:28.045179 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633f1828cf87496fbc0dad682bf9986b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.053464 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633f1828cf87496fbc0dad682bf9986b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.071305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633f1828cf87496fbc0dad682bf9986b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.078214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633f1828cf87496fbc0dad682bf9986b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.078943 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633f1828cf87496fbc0dad682bf9986b'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.085392 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1447/2925] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:03:27 2022] DELETE /identity/v3/users/633f1828cf87496fbc0dad682bf9986b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:03:28.268030 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:28.283578 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624ab4-a7c5-4db0-9699-c051790a957c service swift] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:28.340939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:28.341515 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:28.341791 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:28.342062 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] PATH_INFO: `/v3/projects/df7a0ee4a83f48d78f1c941172cacdb9` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:28.343149 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7a0ee4a83f48d78f1c941172cacdb9)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:28.345048 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:28.391448 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7a0ee4a83f48d78f1c941172cacdb9', 'request_id': 'req-9d770d93-3f0f-480e-8b48-533340d929d4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.397176 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7a0ee4a83f48d78f1c941172cacdb9', 'request_id': 'req-9d770d93-3f0f-480e-8b48-533340d929d4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.403475 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7a0ee4a83f48d78f1c941172cacdb9', 'request_id': 'req-9d770d93-3f0f-480e-8b48-533340d929d4'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.417575 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] The token cache is being invalidate because project df7a0ee4a83f48d78f1c941172cacdb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:03:28.417873 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.418533 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.420223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1479/2926] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:03:28 2022] DELETE /identity/v3/projects/df7a0ee4a83f48d78f1c941172cacdb9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:03:28.424259 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:28.425724 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:28.439503 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfcea-bd87-4b3c-912a-aaa29f6dad96 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:28.460427 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:28.501999 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:28.502747 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:28.503010 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:28.503339 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] PATH_INFO: `/v3/users/94127ba08f9a41deb75c1047359aedf0` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:28.504525 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94127ba08f9a41deb75c1047359aedf0)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:28.506492 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:28.552103 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94127ba08f9a41deb75c1047359aedf0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.560875 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94127ba08f9a41deb75c1047359aedf0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.573083 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94127ba08f9a41deb75c1047359aedf0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.579793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94127ba08f9a41deb75c1047359aedf0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.580454 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94127ba08f9a41deb75c1047359aedf0'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.586295 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1448/2927] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:03:28 2022] DELETE /identity/v3/users/94127ba08f9a41deb75c1047359aedf0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:03:28.757814 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:28.759280 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:28.776437 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d770d93-3f0f-480e-8b48-533340d929d4 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:28.793896 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:28.835214 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:28.835852 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:28.836133 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:28.836406 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] PATH_INFO: `/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:28.837539 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce30a91271b4c2aaf47560f5d2e65b3)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:28.839450 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:28.884231 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce30a91271b4c2aaf47560f5d2e65b3', 'request_id': 'req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.889677 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce30a91271b4c2aaf47560f5d2e65b3', 'request_id': 'req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.895654 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce30a91271b4c2aaf47560f5d2e65b3', 'request_id': 'req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.908705 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] The token cache is being invalidate because project cce30a91271b4c2aaf47560f5d2e65b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:03:28.909019 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.909649 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:03:28.911080 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1480/2928] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:03:28 2022] DELETE /identity/v3/projects/cce30a91271b4c2aaf47560f5d2e65b3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:03:38.582813 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:38.905680 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:31.147124 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:31.148637 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:31.297770 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701bbfb-6c9e-42f5-88e0-11e27656e478 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:31.361629 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:31.362171 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:31.362453 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:31.362729 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:31.364489 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:31.376655 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:31.378257 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e19b7-d383-470a-be11-9c7bad4cef99 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:31.388948 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1449/2929] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:32.744347 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:32.745783 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:32.826037 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79947c49-2b18-4e0f-b2f9-1cb8a55797c0 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:32.829703 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:32.830302 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:32.830585 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:32.830892 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:32.832602 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:32.841985 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:32.843400 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979fb76-4cff-468e-af64-9b52084d1d17 service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:32.851311 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1481/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:32.973055 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d99b3-ddd8-4bc0-9eb1-2768e34d05c8 None None] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:32.973730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d99b3-ddd8-4bc0-9eb1-2768e34d05c8 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:32.973730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d99b3-ddd8-4bc0-9eb1-2768e34d05c8 None None] PATH_INFO: `/` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:32.975214 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1450/2931] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 10:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:05:32.981105 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ad8b-d960-454d-9f4b-3778034821ba None None] REQUEST_METHOD: `POST` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:32.981389 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ad8b-d960-454d-9f4b-3778034821ba None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:32.981681 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ad8b-d960-454d-9f4b-3778034821ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:33.030538 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.auth.core [None req-faf5ad8b-d960-454d-9f4b-3778034821ba None None] MFA Rules not processed for user `87ebde2ec14c40bab8a795230a1c5b42`. Rule list: `[]` (Enabled: `True`). {{(pid=88242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:33.031755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-faf5ad8b-d960-454d-9f4b-3778034821ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:33.038511 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1482/2932] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 10:05:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:05:33.051357 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d99b3-ddd8-4bc0-9eb1-2768e34d05c8 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:33.062907 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d99b3-ddd8-4bc0-9eb1-2768e34d05c8 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:33.067067 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:33.067763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:33.068050 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:33.068327 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:33.077744 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:33.079177 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2280-a0fc-4cb9-97ac-1b80e88945d9 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:33.090942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1451/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:34.181250 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb899a-2bc2-43f9-a16a-11e5bea1814e None None] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:34.181755 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb899a-2bc2-43f9-a16a-11e5bea1814e None None] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:34.181871 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb899a-2bc2-43f9-a16a-11e5bea1814e None None] PATH_INFO: `/` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:34.183229 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1483/2934] 10.222.0.25 () {54 vars in 921 bytes} [Wed Aug 31 10:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:05:34.189277 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] REQUEST_METHOD: `POST` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:34.189730 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:34.189848 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:34.209865 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=100 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:34.228500 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.auth.core [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] MFA Rules not processed for user `ad5260213b9f4ec49f3ce2ec48945d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88241) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:34.229749 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:34.297385 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1452/2935] 10.222.0.25 () {58 vars in 1018 bytes} [Wed Aug 31 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:34.308267 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb899a-2bc2-43f9-a16a-11e5bea1814e None None] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:34.319861 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb899a-2bc2-43f9-a16a-11e5bea1814e None None] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:34.323344 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:34.323941 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:34.324223 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:34.324500 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:34.335653 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:34.338290 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:34.352840 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1484/2936] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:05:34.412990 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:34.414377 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:34.429793 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6832c9-ecf7-48bd-b03e-751fe0c4b2d7 None None] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:34.434731 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:34.435361 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:34.435663 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:34.435942 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:34.438175 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:34.449533 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:34.451126 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:34.460869 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1453/2937] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:34.955107 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:34.957106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:34.974423 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb2b6c-2faa-404d-80c5-9eaa40a6f85b service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:34.980024 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:34.980634 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:34.980920 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:34.981228 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:34.982986 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:34.995935 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:34.997566 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:35.008726 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1485/2938] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:44.456630 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=38 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:45.004189 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=62 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:30.656118 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:30.665808 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91985f-3d5c-4aeb-ad95-fb591a894458 service placement] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:30.669434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:30.670029 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:30.670310 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:30.670584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:30.672333 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:30.730438 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:30.731854 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:30.741708 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1454/2939] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:31.815882 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:31.849806 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e37c-578e-4371-a6f6-cf347aee0d9e service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:31.856435 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5260213b9f4ec49f3ce2ec48945d7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:31.857175 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:31.857543 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:31.857885 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:31.859708 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:31.887355 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:31.889939 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:31.900150 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1486/2940] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:32.036497 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:32.037981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:32.050562 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019aa1-ec30-432f-a18e-b92b24097d5d service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:32.054250 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:32.054883 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:32.055162 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:32.055433 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:32.057207 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:32.070675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:32.072364 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:32.087675 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1455/2941] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:33.271623 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:33.281362 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dc989-c33f-4c80-a4ca-7884824e005e service nova] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:33.284889 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:33.285528 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:33.285809 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:33.286085 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:33.287967 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:33.302556 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:33.303996 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:33.315833 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1487/2942] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:36.982088 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:36.991125 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae23f0-578a-48a2-8436-75716b4c0baf service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:36.995422 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87ebde2ec14c40bab8a795230a1c5b42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:36.995968 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:36.996253 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:36.996529 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:36.998307 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:37.082806 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:37.084273 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:37.103102 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1456/2943] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:40.184540 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:40.186567 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:40.201397 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798108-9f03-4770-9a10-08aa4fef8a0e service neutron] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:40.207119 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:40.207683 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] REQUEST_METHOD: `GET` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:40.207965 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:40.208240 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:40.210011 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:40.225923 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:40.228946 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:40.244830 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1488/2944] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:43.399763 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:43.401226 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:43.414993 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91cb18-65c7-4caa-9aec-bfc8db516966 service neutron] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:43.419668 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:43.420305 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:43.420584 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:43.420858 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] PATH_INFO: `/v3/users/e170e5057c6f4467b247b2175d2b3ba5` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:43.422045 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e170e5057c6f4467b247b2175d2b3ba5)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:43.423947 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:43.475016 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:43.477336 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e170e5057c6f4467b247b2175d2b3ba5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.485194 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e170e5057c6f4467b247b2175d2b3ba5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.497519 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e170e5057c6f4467b247b2175d2b3ba5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.504360 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e170e5057c6f4467b247b2175d2b3ba5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.505034 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e170e5057c6f4467b247b2175d2b3ba5'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.510840 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1457/2945] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:07:43 2022] DELETE /identity/v3/users/e170e5057c6f4467b247b2175d2b3ba5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:07:43.691770 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:43.704972 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8b833-ff39-402b-ac3e-761519e51933 service placement] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:43.763109 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:43.763729 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:43.764007 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:43.764276 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] PATH_INFO: `/v3/projects/52d82faa2ab445e18d6e3d668821c1e5` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:43.765409 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d82faa2ab445e18d6e3d668821c1e5)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:43.767284 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:43.813689 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d82faa2ab445e18d6e3d668821c1e5', 'request_id': 'req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.819127 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d82faa2ab445e18d6e3d668821c1e5', 'request_id': 'req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.826904 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d82faa2ab445e18d6e3d668821c1e5', 'request_id': 'req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.839547 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] The token cache is being invalidate because project 52d82faa2ab445e18d6e3d668821c1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:07:43.839837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.840542 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.842067 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1489/2946] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:07:43 2022] DELETE /identity/v3/projects/52d82faa2ab445e18d6e3d668821c1e5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:07:43.846159 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:43.847627 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:43.861515 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41187d9c-5aa8-419e-8d9e-cbf2bc740004 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:43.918526 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:43.919151 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:43.919435 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:43.919704 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] PATH_INFO: `/v3/users/47d11add4eae4d4986fc8288c22af614` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:43.920804 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d11add4eae4d4986fc8288c22af614)` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:43.922759 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:43.968913 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d11add4eae4d4986fc8288c22af614'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.976839 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d11add4eae4d4986fc8288c22af614'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.988611 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d11add4eae4d4986fc8288c22af614'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.995434 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d11add4eae4d4986fc8288c22af614'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:43.996084 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.notifications [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d11add4eae4d4986fc8288c22af614'} {{(pid=88241) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.001938 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1458/2947] 10.222.0.25 () {58 vars in 1224 bytes} [Wed Aug 31 10:07:43 2022] DELETE /identity/v3/users/47d11add4eae4d4986fc8288c22af614 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:07:44.174264 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Authenticating user token {{(pid=88242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:44.175717 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.fernet_utils [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:44.188394 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5397c-bdf8-4d82-b9ad-22d83081a3c3 admin admin] Validating token access rules against request {{(pid=88242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:44.208837 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:44.248417 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c3232bfea3b429697a9f4a4bfb3705a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a4dee0a684e749e89f40e4287b799', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:44.248983 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:44.249275 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] SCRIPT_NAME: `/identity` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:44.249550 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] PATH_INFO: `/v3/projects/71e85f6e576745cfa30b47b11a9b2f01` {{(pid=88242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:44.250667 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e85f6e576745cfa30b47b11a9b2f01)` {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:44.252596 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] RBAC: Authorization granted {{(pid=88242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:44.298662 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e85f6e576745cfa30b47b11a9b2f01', 'request_id': 'req-e6116b00-969f-435c-aa58-294021e20826'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.304091 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e85f6e576745cfa30b47b11a9b2f01', 'request_id': 'req-e6116b00-969f-435c-aa58-294021e20826'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.310106 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e85f6e576745cfa30b47b11a9b2f01', 'request_id': 'req-e6116b00-969f-435c-aa58-294021e20826'} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.322424 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] The token cache is being invalidate because project 71e85f6e576745cfa30b47b11a9b2f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:07:44.322712 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.323339 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG keystone.notifications [None req-e6116b00-969f-435c-aa58-294021e20826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:07:44.324789 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: [pid: 88242|app: 0|req: 1490/2948] 10.222.0.25 () {58 vars in 1233 bytes} [Wed Aug 31 10:07:44 2022] DELETE /identity/v3/projects/71e85f6e576745cfa30b47b11a9b2f01 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:07:53.998441 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=66,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:54.319051 dv-cinder-837542-8-iscsi devstack@keystone.service[88242]: DEBUG dbcounter [-] [88242] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:10:38.312240 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Authenticating user token {{(pid=88241) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:10:38.313670 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:10:38.382595 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90725ce-ac64-4c31-aa45-a95b3976ac89 admin admin] Validating token access rules against request {{(pid=88241) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:10:38.443081 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be582810785a49d48b0a06cd66b316a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a25e6d820824e73b621c2c157e75383', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88241) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:10:38.443695 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] REQUEST_METHOD: `GET` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:10:38.443981 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] SCRIPT_NAME: `/identity` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:10:38.444260 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88241) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:10:38.446025 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.fernet_utils [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88241) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:10:38.457104 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:10:38.459155 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce81dc5-5e97-4efd-ab88-c45d05cc3c93 service placement] RBAC: Authorization granted {{(pid=88241) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:10:38.469018 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: [pid: 88241|app: 0|req: 1459/2949] 10.222.0.25 () {60 vars in 1365 bytes} [Wed Aug 31 10:10:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:10:48.464784 dv-cinder-837542-8-iscsi devstack@keystone.service[88241]: DEBUG dbcounter [-] [88241] Writing DB stats keystone:SELECT=58 {{(pid=88241) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}